Jul 16 21:37:04.389367 ip-10-0-136-44 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 21:37:04.389379 ip-10-0-136-44 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 21:37:04.389385 ip-10-0-136-44 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 21:37:04.389663 ip-10-0-136-44 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 21:37:14.536569 ip-10-0-136-44 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 21:37:14.536584 ip-10-0-136-44 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d7cfe1bad8234b879dcc9d86437b7b5c -- Jul 16 21:38:54.097838 ip-10-0-136-44 systemd[1]: Starting Kubernetes Kubelet... Jul 16 21:38:54.432781 ip-10-0-136-44 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 21:38:54.432781 ip-10-0-136-44 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 21:38:54.432781 ip-10-0-136-44 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 21:38:54.432781 ip-10-0-136-44 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 21:38:54.432781 ip-10-0-136-44 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 21:38:54.434393 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.434234 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440347 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440367 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440373 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440377 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440380 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440384 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:38:54.440378 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440388 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440392 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440396 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440400 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440403 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440408 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440412 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440415 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440419 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440423 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440427 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440430 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440434 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440438 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440441 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440445 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440449 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440452 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440456 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440461 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:38:54.440769 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440465 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440468 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440473 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440476 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440480 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440483 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440487 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440490 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440495 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440498 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440502 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440506 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440510 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440515 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440520 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440525 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440529 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440533 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440537 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440541 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:38:54.441594 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440546 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440550 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440570 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440576 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440580 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440585 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440589 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440593 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440597 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440603 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440608 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440614 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440620 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440626 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440633 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440640 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440644 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440649 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440653 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:38:54.442494 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440658 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440662 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440667 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440672 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440676 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440681 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440685 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440690 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440694 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440698 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440702 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440707 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440711 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440715 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440719 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440723 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440727 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440731 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440737 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440741 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:38:54.443315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440745 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440749 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440755 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440759 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440764 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440768 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440771 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440775 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440780 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440784 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440787 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440792 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440798 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.440805 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.441972 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.441981 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.441986 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.441990 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.441995 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442000 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:38:54.444200 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442005 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442009 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442013 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442018 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442023 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442027 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442031 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442035 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442039 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442043 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442047 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442053 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442058 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442062 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442067 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442071 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442076 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442080 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442084 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442089 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:38:54.445066 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442093 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442097 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442101 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442105 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442110 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442114 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442118 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442122 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442126 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442130 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442135 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442139 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442143 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442147 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442152 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442156 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442160 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442164 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442168 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442172 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:38:54.445710 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442176 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442180 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442185 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442189 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442194 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442198 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442203 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442208 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442212 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442217 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442221 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442225 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442229 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442233 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442238 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442242 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442246 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442250 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442254 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442258 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:38:54.446292 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442263 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442268 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442272 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442276 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442280 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442284 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442288 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442292 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442296 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442301 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442305 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442309 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442313 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442317 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442321 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442326 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442332 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442339 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442344 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:38:54.447038 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442349 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442354 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442358 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442364 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442370 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442375 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442380 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442385 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442389 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442395 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442400 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442404 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442409 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.442413 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443021 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443030 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443034 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443039 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443043 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443047 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:38:54.447537 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443051 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443056 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443060 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443064 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443067 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443072 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443077 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443081 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443085 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443089 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443094 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443098 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443103 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443108 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443112 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443116 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443120 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443124 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443129 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443133 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:38:54.448092 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443137 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443141 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443145 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443148 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443152 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443157 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443161 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443165 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443170 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443174 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443178 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443182 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443185 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443190 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443194 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443198 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443202 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443206 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443210 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443214 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:38:54.448663 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443218 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443226 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443232 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443237 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443247 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443252 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443257 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443262 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443267 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443271 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443275 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443279 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443283 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443288 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443292 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443296 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443300 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443305 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443309 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:38:54.449153 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443313 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443317 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443321 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443325 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443329 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443332 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443337 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443341 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443344 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443349 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443353 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443357 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443361 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443365 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443369 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443374 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443380 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443387 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443392 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:38:54.449632 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443397 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443404 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443408 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443412 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443416 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443420 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443424 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443428 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443432 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443437 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443441 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443450 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443456 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443461 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.443465 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443585 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443596 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443606 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443612 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443619 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443623 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 21:38:54.450090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443630 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443636 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443641 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443645 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443651 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443656 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443661 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443666 2578 flags.go:64] FLAG: --cgroup-root="" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443670 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443677 2578 flags.go:64] FLAG: --client-ca-file="" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443682 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443687 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443694 2578 flags.go:64] FLAG: --cluster-domain="" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443698 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443703 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443708 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443713 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443719 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443723 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443727 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443733 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443738 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443744 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443749 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 21:38:54.450617 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443754 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443759 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443765 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443769 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443774 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443778 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443783 2578 flags.go:64] FLAG: --enable-server="true" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443787 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443795 2578 flags.go:64] FLAG: --event-burst="100" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443800 2578 flags.go:64] FLAG: --event-qps="50" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443804 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443809 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443813 2578 flags.go:64] FLAG: --eviction-hard="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443819 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443824 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443828 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443833 2578 flags.go:64] FLAG: --eviction-soft="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443839 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443844 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443849 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443854 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443859 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443863 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443867 2578 flags.go:64] FLAG: --feature-gates="" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443873 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 21:38:54.451253 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443878 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443883 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443888 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443893 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443898 2578 flags.go:64] FLAG: --help="false" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443904 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443909 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443914 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443918 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443924 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443929 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443934 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443938 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443942 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443947 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443951 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443956 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443961 2578 flags.go:64] FLAG: --kube-reserved="" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443965 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443970 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443974 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443979 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443983 2578 flags.go:64] FLAG: --lock-file="" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443987 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 21:38:54.451925 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443993 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.443998 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444007 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444012 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444016 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444020 2578 flags.go:64] FLAG: --logging-format="text" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444025 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444030 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444034 2578 flags.go:64] FLAG: --manifest-url="" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444039 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444045 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444050 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444056 2578 flags.go:64] FLAG: --max-pods="110" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444062 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444067 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444071 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444076 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444080 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444085 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444090 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444100 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444105 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444110 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 21:38:54.452510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444114 2578 flags.go:64] FLAG: --pod-cidr="" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444119 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444127 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444131 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444135 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444140 2578 flags.go:64] FLAG: --port="10250" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444145 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444149 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0defbe06dd3e57aba" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444154 2578 flags.go:64] FLAG: --qos-reserved="" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444161 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444166 2578 flags.go:64] FLAG: --register-node="true" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444170 2578 flags.go:64] FLAG: --register-with-taints="" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444184 2578 flags.go:64] FLAG: --registry-burst="10" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444190 2578 flags.go:64] FLAG: --registry-qps="5" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444194 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444199 2578 flags.go:64] FLAG: --reserved-memory="" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444204 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444208 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444212 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444216 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444220 2578 flags.go:64] FLAG: --runonce="false" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444225 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444229 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444236 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444241 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 21:38:54.453086 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444246 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444250 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444255 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444260 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444264 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444269 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444274 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444278 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444282 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444287 2578 flags.go:64] FLAG: --system-cgroups="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444291 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444300 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444304 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444308 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444315 2578 flags.go:64] FLAG: --tls-min-version="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444320 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444325 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444330 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444335 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444339 2578 flags.go:64] FLAG: --v="2" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444345 2578 flags.go:64] FLAG: --version="false" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444352 2578 flags.go:64] FLAG: --vmodule="" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444358 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.444363 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444584 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:38:54.453720 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444592 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444597 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444602 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444606 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444611 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444617 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444622 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444626 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444630 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444634 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444638 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444642 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444647 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444651 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444655 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444659 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444663 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444667 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444671 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444676 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:38:54.454315 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444679 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444683 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444687 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444693 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444697 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444701 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444708 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444713 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444719 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444726 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444730 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444734 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444738 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444742 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444746 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444750 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444756 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444763 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444768 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:38:54.454829 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444772 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444776 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444780 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444784 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444789 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444793 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444797 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444802 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444806 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444810 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444815 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444818 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444822 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444826 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444831 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444835 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444841 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444845 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444849 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444853 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:38:54.455274 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444857 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444861 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444865 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444870 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444874 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444879 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444883 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444887 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444891 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444895 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444901 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444905 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444909 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444913 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444918 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444922 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444926 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444930 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444934 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444938 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:38:54.455806 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444942 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444946 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444951 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444955 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444959 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444963 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444967 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444971 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444977 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444981 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444988 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444993 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.444998 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.445002 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.445006 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.445010 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.445015 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.445019 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:38:54.456327 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.445023 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.445677 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.452606 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.452623 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.452641 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.452647 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452700 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452706 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452711 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452714 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452717 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452720 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452722 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452725 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:38:54.456792 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452728 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452730 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452733 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452735 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452738 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452741 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452743 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452746 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452749 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452751 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452754 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452757 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452759 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452761 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452764 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452767 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452769 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452771 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452774 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:38:54.457162 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452777 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452780 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452785 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452789 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452792 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452794 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452797 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452799 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452802 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452805 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452807 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452810 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452812 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452815 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452817 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452820 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452822 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452825 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452827 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452829 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:38:54.457645 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452832 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452834 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452836 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452839 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452841 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452844 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452846 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452848 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452851 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452853 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452856 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452858 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452860 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452863 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452866 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452869 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452872 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452874 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452877 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452879 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:38:54.458139 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452882 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452886 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452889 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452891 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452894 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452896 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452898 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452901 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452903 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452906 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452908 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452911 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452913 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452915 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452918 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452920 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452922 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452925 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452927 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452930 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:38:54.458719 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452932 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452934 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452937 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452939 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452942 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452944 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452947 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452950 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452953 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452955 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452958 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.452960 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.452965 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453069 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453073 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453076 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:38:54.459239 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453079 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453081 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453084 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453088 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453092 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453095 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453098 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453101 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453104 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453106 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453109 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453112 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453114 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453117 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453119 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453122 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453124 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453127 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453129 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:38:54.459648 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453132 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453134 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453137 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453140 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453143 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453146 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453149 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453152 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453154 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453156 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453159 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453161 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453164 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453166 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453169 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453172 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453176 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453180 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453183 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:38:54.460108 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453185 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453188 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453191 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453194 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453197 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453199 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453201 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453204 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453206 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453208 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453211 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453213 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453215 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453218 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453220 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453223 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453225 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453228 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453230 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453233 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:38:54.460584 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453236 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453238 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453241 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453243 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453246 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453248 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453251 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453253 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453255 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453258 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453260 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453263 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453265 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453267 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453270 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453272 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453274 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453277 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453279 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453282 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:38:54.461151 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453284 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453286 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453289 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453291 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453293 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453296 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453298 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453300 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453303 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453305 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453308 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453310 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453313 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453316 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453319 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453321 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453323 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:38:54.461653 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:54.453326 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.453331 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.453422 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.455633 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.456879 2578 server.go:1019] "Starting client certificate rotation" Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.456978 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.457010 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.458904 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 21:38:54.462065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.460901 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 21:38:54.470029 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.470008 2578 log.go:25] "Validated CRI v1 runtime API" Jul 16 21:38:54.472460 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.472444 2578 log.go:25] "Validated CRI v1 image API" Jul 16 21:38:54.473285 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.473269 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 21:38:54.477940 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.477920 2578 fs.go:135] Filesystem UUIDs: map[689e1196-619e-426e-85c7-eca2efad3545:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bdfd627c-af1c-440e-80f6-23742fccc188:/dev/nvme0n1p4] Jul 16 21:38:54.478025 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.477940 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 21:38:54.482924 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.482815 2578 manager.go:216] Machine: {Timestamp:2026-07-16 21:38:54.481491455 +0000 UTC m=+0.282378118 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100879 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2368264756c20c892dc51d6f62752c SystemUUID:ec236826-4756-c20c-892d-c51d6f62752c BootID:d7cfe1ba-d823-4b87-9dcc-9d86437b7b5c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:38:c8:53:cd:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:38:c8:53:cd:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:c5:89:43:87:34 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 21:38:54.482924 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.482912 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 21:38:54.483075 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.483005 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 21:38:54.486380 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.486355 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 21:38:54.486510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.486386 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-44.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 21:38:54.486978 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.486967 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 21:38:54.486978 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.486982 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 21:38:54.487053 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.486990 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 21:38:54.487566 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.487541 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 21:38:54.488996 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.488986 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 21:38:54.489744 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.489727 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 21:38:54.492883 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.492871 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 16 21:38:54.492998 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.492988 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 21:38:54.493032 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493020 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 16 21:38:54.493058 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493031 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 21:38:54.493058 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493046 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 21:38:54.493058 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493054 2578 kubelet.go:398] "Adding apiserver pod source" Jul 16 21:38:54.493138 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493062 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 21:38:54.493394 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493381 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 21:38:54.493442 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.493401 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 21:38:54.495374 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.495359 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 21:38:54.496670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.496655 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 21:38:54.496756 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.496693 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 21:38:54.497270 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497246 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 21:38:54.497270 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497271 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497278 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497284 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497294 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497299 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497306 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497311 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497319 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497328 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497336 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 21:38:54.497384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.497351 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 21:38:54.498847 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.498834 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 21:38:54.498847 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.498848 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 21:38:54.501898 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.501882 2578 server.go:1268] "Started kubelet" Jul 16 21:38:54.502096 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.501984 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 21:38:54.502191 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.502165 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 21:38:54.502385 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.502357 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 21:38:54.502698 ip-10-0-136-44 systemd[1]: Started Kubernetes Kubelet. Jul 16 21:38:54.503726 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.503710 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 21:38:54.504190 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.504151 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-44.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 21:38:54.504445 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.504403 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 21:38:54.504732 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.504549 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 21:38:54.506101 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.506082 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 16 21:38:54.509395 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.509375 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 21:38:54.510953 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.510202 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-44.ec2.internal.18c2e2f2b8c2856e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-44.ec2.internal,UID:ip-10-0-136-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-44.ec2.internal,},FirstTimestamp:2026-07-16 21:38:54.501856622 +0000 UTC m=+0.302743264,LastTimestamp:2026-07-16 21:38:54.501856622 +0000 UTC m=+0.302743264,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-44.ec2.internal,}" Jul 16 21:38:54.513009 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.512991 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 21:38:54.513009 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513000 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 21:38:54.513145 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.513095 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:54.513520 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513502 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 21:38:54.513520 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513515 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 21:38:54.513653 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513528 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 21:38:54.513653 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513633 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 16 21:38:54.513653 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513643 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 16 21:38:54.513911 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513874 2578 factory.go:55] Registering systemd factory Jul 16 21:38:54.513911 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.513894 2578 factory.go:223] Registration of the systemd container factory successfully Jul 16 21:38:54.514148 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.514070 2578 factory.go:153] Registering CRI-O factory Jul 16 21:38:54.514148 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.514080 2578 factory.go:223] Registration of the crio container factory successfully Jul 16 21:38:54.514148 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.514137 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 21:38:54.514273 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.514154 2578 factory.go:103] Registering Raw factory Jul 16 21:38:54.514273 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.514164 2578 manager.go:1196] Started watching for new ooms in manager Jul 16 21:38:54.514642 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.514618 2578 manager.go:349] Starting recovery of all containers Jul 16 21:38:54.522742 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.522711 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 21:38:54.523005 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.522979 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 21:38:54.524047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.524030 2578 manager.go:354] Recovery completed Jul 16 21:38:54.528688 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.528676 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.530232 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530215 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.530298 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530247 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.530298 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530260 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.530818 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530800 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 21:38:54.530818 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530816 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 21:38:54.530928 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530827 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.530928 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.530831 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 16 21:38:54.531426 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.531413 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.531473 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.531438 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.531473 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.531448 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.533015 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.533000 2578 policy_none.go:49] "None policy: Start" Jul 16 21:38:54.533087 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.533019 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 21:38:54.533087 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.533034 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 21:38:54.535097 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.535084 2578 policy_none.go:47] "Start" Jul 16 21:38:54.542977 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.542822 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-44.ec2.internal.18c2e2f2ba737c8b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-44.ec2.internal,UID:ip-10-0-136-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-44.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-44.ec2.internal,},FirstTimestamp:2026-07-16 21:38:54.530231435 +0000 UTC m=+0.331118077,LastTimestamp:2026-07-16 21:38:54.530231435 +0000 UTC m=+0.331118077,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-44.ec2.internal,}" Jul 16 21:38:54.554747 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.554688 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-44.ec2.internal.18c2e2f2ba73d4a6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-44.ec2.internal,UID:ip-10-0-136-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-44.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-44.ec2.internal,},FirstTimestamp:2026-07-16 21:38:54.53025399 +0000 UTC m=+0.331140631,LastTimestamp:2026-07-16 21:38:54.53025399 +0000 UTC m=+0.331140631,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-44.ec2.internal,}" Jul 16 21:38:54.557038 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.557002 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w47jl" Jul 16 21:38:54.563243 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.563225 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w47jl" Jul 16 21:38:54.566415 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.566341 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-44.ec2.internal.18c2e2f2ba73fc8b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-44.ec2.internal,UID:ip-10-0-136-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-44.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-44.ec2.internal,},FirstTimestamp:2026-07-16 21:38:54.530264203 +0000 UTC m=+0.331150844,LastTimestamp:2026-07-16 21:38:54.530264203 +0000 UTC m=+0.331150844,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-44.ec2.internal,}" Jul 16 21:38:54.568345 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568329 2578 manager.go:337] "Starting Device Plugin manager" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.568368 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568382 2578 server.go:87] "Starting device plugin registration server" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568660 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568673 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568792 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568861 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.568892 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.569371 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 21:38:54.596135 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.569407 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:54.667477 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.667431 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 21:38:54.668750 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.668731 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 21:38:54.668750 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.668738 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.668916 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.668773 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 21:38:54.668916 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.668799 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 21:38:54.668916 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.668838 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 21:38:54.669824 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.669805 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.669936 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.669835 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.669936 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.669847 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.669936 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.669869 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.671333 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.671314 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:38:54.682759 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.682744 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.682870 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.682764 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-44.ec2.internal\": node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:54.706629 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.706611 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:54.769808 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.769791 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal"] Jul 16 21:38:54.769903 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.769845 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.774633 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.774619 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.774718 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.774652 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.774718 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.774666 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.776979 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.776964 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.777102 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.777085 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.777149 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.777114 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.778290 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.778275 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.778360 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.778296 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.778360 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.778306 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.778360 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.778280 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.778449 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.778370 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.778449 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.778381 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.780527 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.780511 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.780627 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.780548 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:54.782285 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.782268 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:54.782371 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.782296 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:54.782371 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.782310 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:54.802161 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.802138 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-44.ec2.internal\" not found" node="ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.806514 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.806497 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-44.ec2.internal\" not found" node="ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.807488 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.807474 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:54.815835 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.815817 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c28dc886d1725c833f72bf7c58394714-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal\" (UID: \"c28dc886d1725c833f72bf7c58394714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.815891 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.815852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28dc886d1725c833f72bf7c58394714-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal\" (UID: \"c28dc886d1725c833f72bf7c58394714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.815891 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.815870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8fa8481daebd4056943f9c05c427dbe3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-44.ec2.internal\" (UID: \"8fa8481daebd4056943f9c05c427dbe3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.908017 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:54.907985 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:54.916381 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.916362 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c28dc886d1725c833f72bf7c58394714-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal\" (UID: \"c28dc886d1725c833f72bf7c58394714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.916429 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.916318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c28dc886d1725c833f72bf7c58394714-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal\" (UID: \"c28dc886d1725c833f72bf7c58394714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.916429 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.916420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28dc886d1725c833f72bf7c58394714-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal\" (UID: \"c28dc886d1725c833f72bf7c58394714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.916492 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.916438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8fa8481daebd4056943f9c05c427dbe3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-44.ec2.internal\" (UID: \"8fa8481daebd4056943f9c05c427dbe3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.916492 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.916472 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8fa8481daebd4056943f9c05c427dbe3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-44.ec2.internal\" (UID: \"8fa8481daebd4056943f9c05c427dbe3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" Jul 16 21:38:54.916549 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:54.916501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28dc886d1725c833f72bf7c58394714-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal\" (UID: \"c28dc886d1725c833f72bf7c58394714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:55.008804 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.008738 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.104241 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.104220 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:55.109461 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.109254 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.110364 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.110349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" Jul 16 21:38:55.210306 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.210273 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.310853 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.310827 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.411306 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.411285 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.456809 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.456793 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 21:38:55.457156 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.456911 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 21:38:55.511840 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.511823 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.571538 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.571466 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 21:33:55 +0000 UTC" deadline="2027-12-23 18:40:13.527184337 +0000 UTC" Jul 16 21:38:55.571538 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.571500 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12597h1m17.955686975s" Jul 16 21:38:55.574619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.574604 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:38:55.575569 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.575538 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:38:55.575648 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.575591 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:38:55.575648 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.575607 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:38:55.602580 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.602543 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:38:55.612872 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.612848 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.713355 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:55.713330 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-44.ec2.internal\" not found" Jul 16 21:38:55.726270 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.726250 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:38:55.742831 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:55.742804 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa8481daebd4056943f9c05c427dbe3.slice/crio-e60ffa46b199124e1ad54f382a7234e43b8aced32b71772d8ac5c6f6d760ab23 WatchSource:0}: Error finding container e60ffa46b199124e1ad54f382a7234e43b8aced32b71772d8ac5c6f6d760ab23: Status 404 returned error can't find the container with id e60ffa46b199124e1ad54f382a7234e43b8aced32b71772d8ac5c6f6d760ab23 Jul 16 21:38:55.743221 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:55.743199 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28dc886d1725c833f72bf7c58394714.slice/crio-a5b66a0ee4cfb363351b88f517645e61b2b6292d31c9289781b5177c005d0ba6 WatchSource:0}: Error finding container a5b66a0ee4cfb363351b88f517645e61b2b6292d31c9289781b5177c005d0ba6: Status 404 returned error can't find the container with id a5b66a0ee4cfb363351b88f517645e61b2b6292d31c9289781b5177c005d0ba6 Jul 16 21:38:55.748768 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.748752 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:38:55.814098 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.814075 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" Jul 16 21:38:55.835022 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.834978 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 21:38:55.836039 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.836027 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" Jul 16 21:38:55.842563 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.842539 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 21:38:55.866088 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:55.866074 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:38:56.494320 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.494284 2578 apiserver.go:52] "Watching apiserver" Jul 16 21:38:56.509245 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.509213 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 21:38:56.510071 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.509624 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-rdcql","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb","openshift-image-registry/node-ca-xd4s2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal","openshift-multus/multus-4wxqq","openshift-multus/network-metrics-daemon-dclch","openshift-network-diagnostics/network-check-target-4cnbb","kube-system/global-pull-secret-syncer-ddq22","kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qx5w5","openshift-dns/node-resolver-pjkrj","openshift-multus/multus-additional-cni-plugins-sb58x","openshift-network-operator/iptables-alerter-xqnlm","openshift-ovn-kubernetes/ovnkube-node-gthgv"] Jul 16 21:38:56.512499 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.512459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.513227 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.513206 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 21:38:56.515920 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.515895 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.516850 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.516830 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.517135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.517116 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.521652 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.518189 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ls9vj\"" Jul 16 21:38:56.521652 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.521475 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.522633 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.522595 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.523520 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.522814 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8x7x5\"" Jul 16 21:38:56.523817 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.522931 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 21:38:56.523941 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.523274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-modprobe-d\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524137 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-systemd\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524291 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-lib-modules\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524393 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8hm4\" (UniqueName: \"kubernetes.io/projected/318c82d8-2edc-435b-a8a5-42225a01f82b-kube-api-access-q8hm4\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.524393 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-kubernetes\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524393 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysctl-d\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524393 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysctl-conf\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524409 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-var-lib-kubelet\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/034f32c7-77dc-44a3-925a-75656ef451c5-etc-tuned\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysconfig\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-sys\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-host\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/034f32c7-77dc-44a3-925a-75656ef451c5-tmp\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/318c82d8-2edc-435b-a8a5-42225a01f82b-iptables-alerter-script\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-run\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw2zx\" (UniqueName: \"kubernetes.io/projected/034f32c7-77dc-44a3-925a-75656ef451c5-kube-api-access-lw2zx\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.524641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/318c82d8-2edc-435b-a8a5-42225a01f82b-host-slash\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.524680 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.523359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.527398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.526796 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.527398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.526832 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.527398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.527253 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5m5wl\"" Jul 16 21:38:56.527676 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.527450 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 21:38:56.529102 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.528741 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 21:38:56.529928 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.529278 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 21:38:56.529928 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.529603 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 21:38:56.529928 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.529839 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.530131 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.530045 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.530475 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.530199 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-78kmx\"" Jul 16 21:38:56.530992 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.530772 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.530992 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.530813 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 21:38:56.531135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.531114 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 21:38:56.531195 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.531151 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.532752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.532733 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:56.532845 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.532818 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:38:56.533414 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.533158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 21:38:56.534099 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.533750 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.534099 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.533774 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.534099 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.533829 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 21:38:56.534099 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.533987 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l682l\"" Jul 16 21:38:56.535625 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.535318 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.535625 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.535400 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:38:56.538234 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.537929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:38:56.538234 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.537997 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:38:56.543533 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.541809 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.545399 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.544663 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.545399 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.544861 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 21:38:56.545399 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.545078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t7bcb\"" Jul 16 21:38:56.545399 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.545243 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.545683 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.545528 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.548678 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.548081 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 21:38:56.548678 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.548077 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:38:56.548678 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.548416 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hnks5\"" Jul 16 21:38:56.550780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.550745 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.550905 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.550883 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 21:38:56.553226 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.553205 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wct5n\"" Jul 16 21:38:56.553450 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.553430 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 21:38:56.556980 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.556958 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 21:38:56.559739 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.559718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.563376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.563339 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 21:38:56.563558 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.563543 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hlz8m\"" Jul 16 21:38:56.564015 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.563666 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 21:38:56.570039 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.570018 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jwdj4" Jul 16 21:38:56.579783 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.579764 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jwdj4" Jul 16 21:38:56.615584 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.615560 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 21:38:56.625555 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.625533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjhpd\" (UniqueName: \"kubernetes.io/projected/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-kube-api-access-mjhpd\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.625757 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.625743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-sys\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.625864 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.625847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-k8s-cni-cncf-io\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.625978 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.625964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/318c82d8-2edc-435b-a8a5-42225a01f82b-iptables-alerter-script\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.626083 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-systemd-units\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.626174 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22066cc8-4543-4e00-ae54-327ba800f77f-ovn-node-metrics-cert\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.626286 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-cni-bin\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.626330 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-conf-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.626362 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-lib-modules\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.626401 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626360 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-os-release\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.626401 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-os-release\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.626464 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8hm4\" (UniqueName: \"kubernetes.io/projected/318c82d8-2edc-435b-a8a5-42225a01f82b-kube-api-access-q8hm4\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.626464 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626436 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-kubelet\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.626566 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626554 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-sys\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.626713 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zj2k\" (UniqueName: \"kubernetes.io/projected/4d457f90-81b9-452c-8a50-1e19bfaf32d4-kube-api-access-6zj2k\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.626840 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626818 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-lib-modules\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.626921 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysconfig\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627018 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/318c82d8-2edc-435b-a8a5-42225a01f82b-iptables-alerter-script\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.627072 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.627072 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.626965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysconfig\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627178 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zps22\" (UniqueName: \"kubernetes.io/projected/a7fc97df-84c8-45e9-af2f-1099dc537363-kube-api-access-zps22\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.627178 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-etc-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.627178 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-cni-binary-copy\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.627178 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1b264886-0acf-4405-b7c4-e352f58819aa-agent-certs\") pod \"konnectivity-agent-rdcql\" (UID: \"1b264886-0acf-4405-b7c4-e352f58819aa\") " pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-systemd\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-registration-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-hosts-file\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z96nm\" (UniqueName: \"kubernetes.io/projected/63a9d655-6a74-4917-ab90-75eec7179b71-kube-api-access-z96nm\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/853ab9b3-3a12-4858-a741-abb04892dd5b-kubelet-config\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-var-lib-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627360 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-ovnkube-config\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.627398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-kubernetes\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysctl-conf\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-var-lib-kubelet\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-cni-binary-copy\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627484 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-kubernetes\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-system-cni-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-etc-kubernetes\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627538 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-var-lib-kubelet\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-host\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/034f32c7-77dc-44a3-925a-75656ef451c5-tmp\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-system-cni-dir\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f870e849-9509-4e3a-9264-0ec11181dfb4-serviceca\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627697 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysctl-conf\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-env-overrides\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-cni-multus\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-host\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.627917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-sys-fs\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-cnibin\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627866 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-ovnkube-script-lib\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-socket-dir-parent\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-device-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvkjb\" (UniqueName: \"kubernetes.io/projected/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-kube-api-access-nvkjb\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.627980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/318c82d8-2edc-435b-a8a5-42225a01f82b-host-slash\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-modprobe-d\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628033 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-systemd\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-run-netns\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-node-log\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-log-socket\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628090 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-netns\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-hostroot\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysctl-d\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.628811 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628217 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/034f32c7-77dc-44a3-925a-75656ef451c5-etc-tuned\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj8gz\" (UniqueName: \"kubernetes.io/projected/f870e849-9509-4e3a-9264-0ec11181dfb4-kube-api-access-dj8gz\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-ovn\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628290 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628442 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-sysctl-d\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-systemd\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/318c82d8-2edc-435b-a8a5-42225a01f82b-host-slash\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628585 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-socket-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-tmp-dir\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628674 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-etc-modprobe-d\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-cni-bin\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-cni-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-cnibin\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:56.629675 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/853ab9b3-3a12-4858-a741-abb04892dd5b-dbus\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-run\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw2zx\" (UniqueName: \"kubernetes.io/projected/034f32c7-77dc-44a3-925a-75656ef451c5-kube-api-access-lw2zx\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.628997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/034f32c7-77dc-44a3-925a-75656ef451c5-run\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629034 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f870e849-9509-4e3a-9264-0ec11181dfb4-host\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629060 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq2jr\" (UniqueName: \"kubernetes.io/projected/22066cc8-4543-4e00-ae54-327ba800f77f-kube-api-access-lq2jr\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629169 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-daemon-config\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-multus-certs\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1b264886-0acf-4405-b7c4-e352f58819aa-konnectivity-ca\") pod \"konnectivity-agent-rdcql\" (UID: \"1b264886-0acf-4405-b7c4-e352f58819aa\") " pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-slash\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-kubelet\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629436 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.630504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.629487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-cni-netd\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.632951 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.632928 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/034f32c7-77dc-44a3-925a-75656ef451c5-etc-tuned\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.633043 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.633006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/034f32c7-77dc-44a3-925a-75656ef451c5-tmp\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.635703 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.635677 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8hm4\" (UniqueName: \"kubernetes.io/projected/318c82d8-2edc-435b-a8a5-42225a01f82b-kube-api-access-q8hm4\") pod \"iptables-alerter-xqnlm\" (UID: \"318c82d8-2edc-435b-a8a5-42225a01f82b\") " pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.638264 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.638219 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw2zx\" (UniqueName: \"kubernetes.io/projected/034f32c7-77dc-44a3-925a-75656ef451c5-kube-api-access-lw2zx\") pod \"tuned-qx5w5\" (UID: \"034f32c7-77dc-44a3-925a-75656ef451c5\") " pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.675929 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.675875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" event={"ID":"8fa8481daebd4056943f9c05c427dbe3","Type":"ContainerStarted","Data":"e60ffa46b199124e1ad54f382a7234e43b8aced32b71772d8ac5c6f6d760ab23"} Jul 16 21:38:56.679088 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.679055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" event={"ID":"c28dc886d1725c833f72bf7c58394714","Type":"ContainerStarted","Data":"a5b66a0ee4cfb363351b88f517645e61b2b6292d31c9289781b5177c005d0ba6"} Jul 16 21:38:56.730346 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-k8s-cni-cncf-io\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.730501 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730362 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-systemd-units\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.730501 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22066cc8-4543-4e00-ae54-327ba800f77f-ovn-node-metrics-cert\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.730501 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-cni-bin\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.730501 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-conf-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.730501 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-os-release\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-os-release\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-kubelet\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zj2k\" (UniqueName: \"kubernetes.io/projected/4d457f90-81b9-452c-8a50-1e19bfaf32d4-kube-api-access-6zj2k\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730637 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zps22\" (UniqueName: \"kubernetes.io/projected/a7fc97df-84c8-45e9-af2f-1099dc537363-kube-api-access-zps22\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-etc-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730731 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-cni-binary-copy\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.730793 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1b264886-0acf-4405-b7c4-e352f58819aa-agent-certs\") pod \"konnectivity-agent-rdcql\" (UID: \"1b264886-0acf-4405-b7c4-e352f58819aa\") " pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-systemd\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-registration-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-hosts-file\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z96nm\" (UniqueName: \"kubernetes.io/projected/63a9d655-6a74-4917-ab90-75eec7179b71-kube-api-access-z96nm\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/853ab9b3-3a12-4858-a741-abb04892dd5b-kubelet-config\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730897 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-var-lib-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-ovnkube-config\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-cni-binary-copy\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-system-cni-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.730985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-etc-kubernetes\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-system-cni-dir\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f870e849-9509-4e3a-9264-0ec11181dfb4-serviceca\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731090 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-env-overrides\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-cni-multus\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-sys-fs\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.731169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-cnibin\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-ovnkube-script-lib\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-socket-dir-parent\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-device-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvkjb\" (UniqueName: \"kubernetes.io/projected/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-kube-api-access-nvkjb\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-run-netns\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-node-log\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-log-socket\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-netns\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-hostroot\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj8gz\" (UniqueName: \"kubernetes.io/projected/f870e849-9509-4e3a-9264-0ec11181dfb4-kube-api-access-dj8gz\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-ovn\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731484 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-socket-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-tmp-dir\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.731887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731560 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-cni-bin\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-cni-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731590 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-cnibin\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/853ab9b3-3a12-4858-a741-abb04892dd5b-dbus\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f870e849-9509-4e3a-9264-0ec11181dfb4-host\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq2jr\" (UniqueName: \"kubernetes.io/projected/22066cc8-4543-4e00-ae54-327ba800f77f-kube-api-access-lq2jr\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-daemon-config\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-multus-certs\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1b264886-0acf-4405-b7c4-e352f58819aa-konnectivity-ca\") pod \"konnectivity-agent-rdcql\" (UID: \"1b264886-0acf-4405-b7c4-e352f58819aa\") " pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731840 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-ovn\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-slash\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-kubelet\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731932 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-cni-netd\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.732735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.731959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjhpd\" (UniqueName: \"kubernetes.io/projected/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-kube-api-access-mjhpd\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732244 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f870e849-9509-4e3a-9264-0ec11181dfb4-serviceca\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732254 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-device-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-env-overrides\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732308 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-k8s-cni-cncf-io\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732325 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-run-netns\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732374 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-conf-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732369 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-systemd-units\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732400 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-socket-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732440 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-log-socket\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-os-release\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732506 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-os-release\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732527 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-cnibin\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-hostroot\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732584 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-kubelet\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.733702 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732325 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-sys-fs\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732794 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-tmp-dir\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732847 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-netns\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732895 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-cni-bin\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733077 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-cni-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733132 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-cnibin\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.733219 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.733285 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:38:57.233263925 +0000 UTC m=+3.034150577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/853ab9b3-3a12-4858-a741-abb04892dd5b-dbus\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733467 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-etc-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f870e849-9509-4e3a-9264-0ec11181dfb4-host\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-ovnkube-script-lib\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733693 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-socket-dir-parent\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-cni-multus\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-var-lib-openvswitch\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.734590 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733916 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-cni-binary-copy\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733973 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-run-systemd\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734023 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4d457f90-81b9-452c-8a50-1e19bfaf32d4-registration-dir\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-hosts-file\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734205 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-multus-daemon-config\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734228 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22066cc8-4543-4e00-ae54-327ba800f77f-ovnkube-config\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734261 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-run-multus-certs\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.734356 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.734403 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:38:57.234386333 +0000 UTC m=+3.035272967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734446 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-cni-netd\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.733218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734522 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/853ab9b3-3a12-4858-a741-abb04892dd5b-kubelet-config\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7fc97df-84c8-45e9-af2f-1099dc537363-cni-binary-copy\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-host-slash\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734601 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-system-cni-dir\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734650 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-etc-kubernetes\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734745 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.735526 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.732442 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22066cc8-4543-4e00-ae54-327ba800f77f-node-log\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.736441 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1b264886-0acf-4405-b7c4-e352f58819aa-konnectivity-ca\") pod \"konnectivity-agent-rdcql\" (UID: \"1b264886-0acf-4405-b7c4-e352f58819aa\") " pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.736441 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734799 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-cni-bin\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.736441 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734843 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7fc97df-84c8-45e9-af2f-1099dc537363-system-cni-dir\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.736441 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.734848 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-host-var-lib-kubelet\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.736441 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.735245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22066cc8-4543-4e00-ae54-327ba800f77f-ovn-node-metrics-cert\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.738498 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.738468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1b264886-0acf-4405-b7c4-e352f58819aa-agent-certs\") pod \"konnectivity-agent-rdcql\" (UID: \"1b264886-0acf-4405-b7c4-e352f58819aa\") " pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.742348 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.742183 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:38:56.742348 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.742222 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:38:56.742348 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.742238 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:38:56.742348 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:56.742321 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:38:57.242303843 +0000 UTC m=+3.043190494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:38:56.742573 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.742551 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvkjb\" (UniqueName: \"kubernetes.io/projected/4a85dd8e-53c8-408d-bd91-1a46d81a43fc-kube-api-access-nvkjb\") pod \"node-resolver-pjkrj\" (UID: \"4a85dd8e-53c8-408d-bd91-1a46d81a43fc\") " pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.746999 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.746971 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj8gz\" (UniqueName: \"kubernetes.io/projected/f870e849-9509-4e3a-9264-0ec11181dfb4-kube-api-access-dj8gz\") pod \"node-ca-xd4s2\" (UID: \"f870e849-9509-4e3a-9264-0ec11181dfb4\") " pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.747179 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.747152 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq2jr\" (UniqueName: \"kubernetes.io/projected/22066cc8-4543-4e00-ae54-327ba800f77f-kube-api-access-lq2jr\") pod \"ovnkube-node-gthgv\" (UID: \"22066cc8-4543-4e00-ae54-327ba800f77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.748797 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.748773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z96nm\" (UniqueName: \"kubernetes.io/projected/63a9d655-6a74-4917-ab90-75eec7179b71-kube-api-access-z96nm\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:56.748902 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.748846 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zps22\" (UniqueName: \"kubernetes.io/projected/a7fc97df-84c8-45e9-af2f-1099dc537363-kube-api-access-zps22\") pod \"multus-additional-cni-plugins-sb58x\" (UID: \"a7fc97df-84c8-45e9-af2f-1099dc537363\") " pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:56.749066 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.749046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zj2k\" (UniqueName: \"kubernetes.io/projected/4d457f90-81b9-452c-8a50-1e19bfaf32d4-kube-api-access-6zj2k\") pod \"aws-ebs-csi-driver-node-h6vpb\" (UID: \"4d457f90-81b9-452c-8a50-1e19bfaf32d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.750264 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.750240 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjhpd\" (UniqueName: \"kubernetes.io/projected/d5b49fd4-2713-41dc-97f8-e3a29f9e14c0-kube-api-access-mjhpd\") pod \"multus-4wxqq\" (UID: \"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0\") " pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.833145 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.833065 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" Jul 16 21:38:56.845922 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.845892 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034f32c7_77dc_44a3_925a_75656ef451c5.slice/crio-e4831e985c7e5a7e818cb8db54a12077cd97116003344fa8737b6ccfa39618d4 WatchSource:0}: Error finding container e4831e985c7e5a7e818cb8db54a12077cd97116003344fa8737b6ccfa39618d4: Status 404 returned error can't find the container with id e4831e985c7e5a7e818cb8db54a12077cd97116003344fa8737b6ccfa39618d4 Jul 16 21:38:56.852489 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.852467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqnlm" Jul 16 21:38:56.857784 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.857758 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xd4s2" Jul 16 21:38:56.862183 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.862158 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318c82d8_2edc_435b_a8a5_42225a01f82b.slice/crio-266bfc6a859b16b441a986543cd9955006289d4e07a1066465b24afe9c2dee8e WatchSource:0}: Error finding container 266bfc6a859b16b441a986543cd9955006289d4e07a1066465b24afe9c2dee8e: Status 404 returned error can't find the container with id 266bfc6a859b16b441a986543cd9955006289d4e07a1066465b24afe9c2dee8e Jul 16 21:38:56.865060 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.865043 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:38:56.869529 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.869502 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf870e849_9509_4e3a_9264_0ec11181dfb4.slice/crio-d382d5fa726a0412a20374843f5cc31d2e548b0c783ad3e300a3687694dd7181 WatchSource:0}: Error finding container d382d5fa726a0412a20374843f5cc31d2e548b0c783ad3e300a3687694dd7181: Status 404 returned error can't find the container with id d382d5fa726a0412a20374843f5cc31d2e548b0c783ad3e300a3687694dd7181 Jul 16 21:38:56.874597 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.874539 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4wxqq" Jul 16 21:38:56.876422 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.876399 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22066cc8_4543_4e00_ae54_327ba800f77f.slice/crio-1eff7c47f97f50308cbb6c2f752d6af9b0cac7ff5b61ea04e930c6e97ce0c6c8 WatchSource:0}: Error finding container 1eff7c47f97f50308cbb6c2f752d6af9b0cac7ff5b61ea04e930c6e97ce0c6c8: Status 404 returned error can't find the container with id 1eff7c47f97f50308cbb6c2f752d6af9b0cac7ff5b61ea04e930c6e97ce0c6c8 Jul 16 21:38:56.882816 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.882793 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" Jul 16 21:38:56.886493 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.886465 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5b49fd4_2713_41dc_97f8_e3a29f9e14c0.slice/crio-86991d16affb4ef2e40dc1fb6cf7b2a04eb9fe65299ea912c457443641cecfe6 WatchSource:0}: Error finding container 86991d16affb4ef2e40dc1fb6cf7b2a04eb9fe65299ea912c457443641cecfe6: Status 404 returned error can't find the container with id 86991d16affb4ef2e40dc1fb6cf7b2a04eb9fe65299ea912c457443641cecfe6 Jul 16 21:38:56.891823 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.891804 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:38:56.893893 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.893866 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d457f90_81b9_452c_8a50_1e19bfaf32d4.slice/crio-ce2d529c722cdc597467f989f08991602d0ac65fe5fb1e41a39084707338c0e1 WatchSource:0}: Error finding container ce2d529c722cdc597467f989f08991602d0ac65fe5fb1e41a39084707338c0e1: Status 404 returned error can't find the container with id ce2d529c722cdc597467f989f08991602d0ac65fe5fb1e41a39084707338c0e1 Jul 16 21:38:56.901888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.901866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pjkrj" Jul 16 21:38:56.902234 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:56.902201 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b264886_0acf_4405_b7c4_e352f58819aa.slice/crio-a8c7d5a88a202908701c2e18a9879d5f446b8950137eff53a224d3e0cbf2f55e WatchSource:0}: Error finding container a8c7d5a88a202908701c2e18a9879d5f446b8950137eff53a224d3e0cbf2f55e: Status 404 returned error can't find the container with id a8c7d5a88a202908701c2e18a9879d5f446b8950137eff53a224d3e0cbf2f55e Jul 16 21:38:56.909481 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:56.909453 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sb58x" Jul 16 21:38:57.236121 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.236040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:57.236121 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.236079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:57.236315 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.236220 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:38:57.236315 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.236224 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:38:57.236315 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.236296 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:38:58.236275168 +0000 UTC m=+4.037161800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:38:57.236463 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.236317 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:38:58.236307318 +0000 UTC m=+4.037193956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:38:57.336794 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.336752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:38:57.336950 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.336931 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:38:57.337011 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.336959 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:38:57.337011 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.336974 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:38:57.337093 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.337037 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:38:58.337019412 +0000 UTC m=+4.137906064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:38:57.583685 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.583641 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:33:57 +0000 UTC" deadline="2028-02-06 03:24:12.094487019 +0000 UTC" Jul 16 21:38:57.583685 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.583681 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13661h45m14.510810319s" Jul 16 21:38:57.669818 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.669788 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:57.669990 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:57.669909 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:38:57.682417 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.682382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xd4s2" event={"ID":"f870e849-9509-4e3a-9264-0ec11181dfb4","Type":"ContainerStarted","Data":"d382d5fa726a0412a20374843f5cc31d2e548b0c783ad3e300a3687694dd7181"} Jul 16 21:38:57.683646 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.683615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rdcql" event={"ID":"1b264886-0acf-4405-b7c4-e352f58819aa","Type":"ContainerStarted","Data":"a8c7d5a88a202908701c2e18a9879d5f446b8950137eff53a224d3e0cbf2f55e"} Jul 16 21:38:57.684999 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.684971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"1eff7c47f97f50308cbb6c2f752d6af9b0cac7ff5b61ea04e930c6e97ce0c6c8"} Jul 16 21:38:57.686239 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.686213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqnlm" event={"ID":"318c82d8-2edc-435b-a8a5-42225a01f82b","Type":"ContainerStarted","Data":"266bfc6a859b16b441a986543cd9955006289d4e07a1066465b24afe9c2dee8e"} Jul 16 21:38:57.687437 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.687407 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" event={"ID":"034f32c7-77dc-44a3-925a-75656ef451c5","Type":"ContainerStarted","Data":"e4831e985c7e5a7e818cb8db54a12077cd97116003344fa8737b6ccfa39618d4"} Jul 16 21:38:57.688664 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.688635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" event={"ID":"4d457f90-81b9-452c-8a50-1e19bfaf32d4","Type":"ContainerStarted","Data":"ce2d529c722cdc597467f989f08991602d0ac65fe5fb1e41a39084707338c0e1"} Jul 16 21:38:57.689892 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:57.689867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4wxqq" event={"ID":"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0","Type":"ContainerStarted","Data":"86991d16affb4ef2e40dc1fb6cf7b2a04eb9fe65299ea912c457443641cecfe6"} Jul 16 21:38:57.991964 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:57.991827 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a85dd8e_53c8_408d_bd91_1a46d81a43fc.slice/crio-40e30b7fe633d5a26a75d0a9822c48bfc4f0a96b51ce81f0f55f0ae2e94c1487 WatchSource:0}: Error finding container 40e30b7fe633d5a26a75d0a9822c48bfc4f0a96b51ce81f0f55f0ae2e94c1487: Status 404 returned error can't find the container with id 40e30b7fe633d5a26a75d0a9822c48bfc4f0a96b51ce81f0f55f0ae2e94c1487 Jul 16 21:38:57.992523 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:38:57.992501 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fc97df_84c8_45e9_af2f_1099dc537363.slice/crio-c92a3e17a2b395f5d63e05390554058d3a29284a71be602bb13b32b93c5dab18 WatchSource:0}: Error finding container c92a3e17a2b395f5d63e05390554058d3a29284a71be602bb13b32b93c5dab18: Status 404 returned error can't find the container with id c92a3e17a2b395f5d63e05390554058d3a29284a71be602bb13b32b93c5dab18 Jul 16 21:38:58.245785 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.244876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:58.245785 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.245186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:58.245785 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.245349 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:38:58.245785 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.245417 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:00.245397737 +0000 UTC m=+6.046284462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:38:58.246222 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.246075 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:38:58.246222 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.246148 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:00.246131587 +0000 UTC m=+6.047018217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:38:58.346020 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.345980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:38:58.346207 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.346157 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:38:58.346207 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.346180 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:38:58.346207 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.346193 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:38:58.346368 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.346257 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:00.34623587 +0000 UTC m=+6.147122518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:38:58.584088 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.584046 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:33:57 +0000 UTC" deadline="2028-04-16 22:06:18.505162512 +0000 UTC" Jul 16 21:38:58.584088 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.584087 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15360h27m19.921081358s" Jul 16 21:38:58.672007 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.671971 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:38:58.672193 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.672128 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:38:58.672500 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.672479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:38:58.672645 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:58.672604 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:38:58.703872 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.703832 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerStarted","Data":"c92a3e17a2b395f5d63e05390554058d3a29284a71be602bb13b32b93c5dab18"} Jul 16 21:38:58.706795 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.706761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pjkrj" event={"ID":"4a85dd8e-53c8-408d-bd91-1a46d81a43fc","Type":"ContainerStarted","Data":"40e30b7fe633d5a26a75d0a9822c48bfc4f0a96b51ce81f0f55f0ae2e94c1487"} Jul 16 21:38:58.716577 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.715692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" event={"ID":"8fa8481daebd4056943f9c05c427dbe3","Type":"ContainerStarted","Data":"8e7f22a9d84b26420103abfa5d14e3eafe4b0bbcf0f0c587ebd692790a746b5f"} Jul 16 21:38:58.722843 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.722812 2578 generic.go:356] "Generic (PLEG): container finished" podID="c28dc886d1725c833f72bf7c58394714" containerID="cecb7247686f71f818a26f2d3ee0778872e9c39386d5637472155672d0f777ae" exitCode=0 Jul 16 21:38:58.722973 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.722853 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" event={"ID":"c28dc886d1725c833f72bf7c58394714","Type":"ContainerDied","Data":"cecb7247686f71f818a26f2d3ee0778872e9c39386d5637472155672d0f777ae"} Jul 16 21:38:58.738723 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:58.738663 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-44.ec2.internal" podStartSLOduration=2.738626714 podStartE2EDuration="2.738626714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:38:58.737763043 +0000 UTC m=+4.538649699" watchObservedRunningTime="2026-07-16 21:38:58.738626714 +0000 UTC m=+4.539513370" Jul 16 21:38:59.669398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:59.669357 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:38:59.670116 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:38:59.669492 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:38:59.758729 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:38:59.758697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" event={"ID":"c28dc886d1725c833f72bf7c58394714","Type":"ContainerStarted","Data":"2ac91e27bef0840a1b8ad5e4e8b43540b34050b848c8e4c417a8922e5f950f01"} Jul 16 21:39:00.262769 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:00.262728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:00.262972 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:00.262789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:00.263035 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.262984 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:00.263096 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.263058 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:04.263037093 +0000 UTC m=+10.063923727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:00.263508 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.263487 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:00.263598 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.263550 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:04.263531089 +0000 UTC m=+10.064417723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:00.363889 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:00.363805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:00.364070 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.363986 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:39:00.364070 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.364010 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:39:00.364070 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.364024 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:00.364246 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.364088 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:04.364067315 +0000 UTC m=+10.164953949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:00.670833 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:00.670753 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:00.671294 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.670912 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:00.671358 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:00.671310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:00.671416 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:00.671402 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:01.669910 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:01.669873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:01.670123 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:01.670011 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:02.669672 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:02.669591 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:02.669672 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:02.669609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:02.670233 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:02.669750 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:02.670299 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:02.670221 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:03.669450 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:03.669412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:03.669673 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:03.669556 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:04.296762 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:04.296715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:04.297268 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:04.296773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:04.297268 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.296909 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:04.297268 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.296968 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:12.296950328 +0000 UTC m=+18.097836962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:04.297386 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.297351 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:04.297421 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.297388 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:12.297376654 +0000 UTC m=+18.098263289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:04.397423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:04.397385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:04.397607 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.397543 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:39:04.397607 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.397567 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:39:04.397607 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.397580 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:04.397768 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.397662 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:12.397622011 +0000 UTC m=+18.198508643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:04.670528 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:04.670442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:04.670681 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.670590 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:04.670681 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:04.670671 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:04.670799 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:04.670781 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:05.669965 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:05.669513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:05.669965 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:05.669641 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:06.672970 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:06.672941 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:06.673424 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:06.672953 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:06.673424 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:06.673066 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:06.673424 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:06.673174 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:07.669133 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:07.669097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:07.669329 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:07.669242 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:08.672639 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:08.672604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:08.672639 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:08.672631 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:08.673128 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:08.672708 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:08.673128 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:08.672852 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:09.669896 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:09.669858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:09.670077 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:09.670020 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:10.672376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:10.672349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:10.672376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:10.672363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:10.672864 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:10.672453 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:10.672864 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:10.672588 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:11.669468 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:11.669436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:11.669700 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:11.669585 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:12.355346 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:12.355310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:12.355806 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:12.355358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:12.355806 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.355469 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:12.355806 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.355498 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:12.355806 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.355528 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:28.355511195 +0000 UTC m=+34.156397832 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:12.355806 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.355577 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:28.355541 +0000 UTC m=+34.156427644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:12.455905 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:12.455868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:12.456067 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.456021 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:39:12.456067 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.456037 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:39:12.456067 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.456045 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:12.456209 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.456091 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:28.456080058 +0000 UTC m=+34.256966686 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:12.669621 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:12.669528 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:12.669769 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.669696 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:12.669769 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:12.669744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:12.669894 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:12.669843 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:13.669335 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:13.669305 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:13.669774 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:13.669413 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:14.670172 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:14.669766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:14.670172 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:14.669863 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:14.670172 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:14.669927 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:14.670172 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:14.670038 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:15.669741 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:15.669702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:15.669927 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:15.669847 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:16.669735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:16.669697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:16.670185 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:16.669844 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:16.670185 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:16.669893 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:16.670185 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:16.669980 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:17.669902 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.669585 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:17.670442 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:17.670005 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:17.800994 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.800963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rdcql" event={"ID":"1b264886-0acf-4405-b7c4-e352f58819aa","Type":"ContainerStarted","Data":"588e2a15daec9fd4f6606b1b1005aaaee161d4af8691c3a7d11bee93dd59f176"} Jul 16 21:39:17.805248 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.805224 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"1b666dcf55256cc1da82a51081589b82d1174eae14b62866faf4b870f9b886de"} Jul 16 21:39:17.805337 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.805299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"1c81c6059c45f71489ccf850e901c4b587b6d225fd7e4accf29c53b587b46dbb"} Jul 16 21:39:17.805390 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.805339 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"c60d4a26e32eed8cd1d67bd96a9b8a2a4d4203e0e59112f2fb1e571aa223b336"} Jul 16 21:39:17.805390 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.805378 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"5dce3f717dfabf546e86610f3e82f0c463fda111df993e75dad162123fd70bc9"} Jul 16 21:39:17.805454 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.805419 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"6f6f801bf207d6479d937f243473c80eac2e04945b9254c4a5484a1802a275ca"} Jul 16 21:39:17.805485 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.805458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"7f7865cd502687b76584a87dc0058b5bd58b6961ccb0ab245f1cfa8b8370ba3d"} Jul 16 21:39:17.807296 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.807273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" event={"ID":"034f32c7-77dc-44a3-925a-75656ef451c5","Type":"ContainerStarted","Data":"b50b044c72abfd401110f198f9907d6cd1103d5a6d81aa8117c4460e48a27065"} Jul 16 21:39:17.808830 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.808810 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7fc97df-84c8-45e9-af2f-1099dc537363" containerID="dcdef5277a39daab0191c2f4b0f9c17919c3b34916c652e36569ec2df10bb8c7" exitCode=0 Jul 16 21:39:17.808929 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.808891 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerDied","Data":"dcdef5277a39daab0191c2f4b0f9c17919c3b34916c652e36569ec2df10bb8c7"} Jul 16 21:39:17.810869 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.810846 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pjkrj" event={"ID":"4a85dd8e-53c8-408d-bd91-1a46d81a43fc","Type":"ContainerStarted","Data":"30addad875220c59e2acebd2fa186f13f213197bc2fa5e2e69799901e1d64224"} Jul 16 21:39:17.812400 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.812374 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" event={"ID":"4d457f90-81b9-452c-8a50-1e19bfaf32d4","Type":"ContainerStarted","Data":"f4bb4921f49438ec3e84248efba8508f08febb153e4482da26d3d78f9c359156"} Jul 16 21:39:17.814263 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.814165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4wxqq" event={"ID":"d5b49fd4-2713-41dc-97f8-e3a29f9e14c0","Type":"ContainerStarted","Data":"7758bf007ad7b0c6d26e1bfd7c71a2c1d6db8c3b64f96b833b6f1e07106f59b0"} Jul 16 21:39:17.815847 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.815826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xd4s2" event={"ID":"f870e849-9509-4e3a-9264-0ec11181dfb4","Type":"ContainerStarted","Data":"0f2cc3fb24c514313704bace8e5c5706cce38dd0ad3484c936e120d65e47b04b"} Jul 16 21:39:17.818238 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.818191 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rdcql" podStartSLOduration=6.862350283 podStartE2EDuration="22.818175664s" totalImagesPullingTime="15.955825381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:17.81786576 +0000 UTC m=+23.618752412" watchObservedRunningTime="2026-07-16 21:39:17.818175664 +0000 UTC m=+23.619062316" Jul 16 21:39:17.818398 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.818374 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-44.ec2.internal" podStartSLOduration=21.818368316 podStartE2EDuration="21.818368316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:38:59.777309578 +0000 UTC m=+5.578196235" watchObservedRunningTime="2026-07-16 21:39:17.818368316 +0000 UTC m=+23.619254966" Jul 16 21:39:17.832618 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.832518 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pjkrj" podStartSLOduration=3.736567279 podStartE2EDuration="22.832501983s" totalImagesPullingTime="19.095934704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:17.832019392 +0000 UTC m=+23.632906044" watchObservedRunningTime="2026-07-16 21:39:17.832501983 +0000 UTC m=+23.633388635" Jul 16 21:39:17.880309 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.880228 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qx5w5" podStartSLOduration=2.636800557 podStartE2EDuration="22.880215223s" totalImagesPullingTime="20.243414666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:17.880101347 +0000 UTC m=+23.680987999" watchObservedRunningTime="2026-07-16 21:39:17.880215223 +0000 UTC m=+23.681101906" Jul 16 21:39:17.895585 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.895524 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xd4s2" podStartSLOduration=2.680386888 podStartE2EDuration="22.895510096s" totalImagesPullingTime="20.215123208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:17.894798231 +0000 UTC m=+23.695684882" watchObservedRunningTime="2026-07-16 21:39:17.895510096 +0000 UTC m=+23.696396747" Jul 16 21:39:17.918026 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:17.917985 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4wxqq" podStartSLOduration=2.661156409 podStartE2EDuration="22.917970168s" totalImagesPullingTime="20.256813759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:17.917395939 +0000 UTC m=+23.718282589" watchObservedRunningTime="2026-07-16 21:39:17.917970168 +0000 UTC m=+23.718856819" Jul 16 21:39:18.669391 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:18.669338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:18.669475 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:18.669438 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:18.669509 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:18.669490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:18.669613 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:18.669595 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:18.704418 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:18.704393 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 21:39:18.819725 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:18.819693 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqnlm" event={"ID":"318c82d8-2edc-435b-a8a5-42225a01f82b","Type":"ContainerStarted","Data":"fd6c516a0b1f7b323741a60f78ba4ded47849587b7ad5c8804cc637f8ba4ba9a"} Jul 16 21:39:18.821345 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:18.821320 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" event={"ID":"4d457f90-81b9-452c-8a50-1e19bfaf32d4","Type":"ContainerStarted","Data":"d00f60a6b6ddf6febe87ed4bb66d8a8bfd43aaaf1d78b2518947008dc98f8037"} Jul 16 21:39:18.835217 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:18.835181 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xqnlm" podStartSLOduration=3.6071856799999997 podStartE2EDuration="23.835168835s" totalImagesPullingTime="20.227983155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:18.835150077 +0000 UTC m=+24.636036730" watchObservedRunningTime="2026-07-16 21:39:18.835168835 +0000 UTC m=+24.636055480" Jul 16 21:39:19.584471 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:19.584350 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T21:39:18.704408228Z","UUID":"51f4a2da-9c49-4e74-9670-c14214864e96","Handler":null,"Name":"","Endpoint":""} Jul 16 21:39:19.586101 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:19.586078 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 21:39:19.586229 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:19.586136 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 21:39:19.669506 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:19.669479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:19.669708 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:19.669626 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:20.669736 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.669710 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:20.670191 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:20.669815 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:20.670191 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.669908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:20.670191 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:20.670015 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:20.829307 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.829134 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"e72d6f6d02fc336f742abf874257568b2f95c27dbe358e2885a39a738060b3de"} Jul 16 21:39:20.830971 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.830950 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" event={"ID":"4d457f90-81b9-452c-8a50-1e19bfaf32d4","Type":"ContainerStarted","Data":"d68d6b47815f77ccce99038d2e7db33002dd6fd16356c4d5490defe2c10e91fa"} Jul 16 21:39:20.854808 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.854680 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6vpb" podStartSLOduration=2.973787885 podStartE2EDuration="25.854656258s" totalImagesPullingTime="22.880868373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:20.853099424 +0000 UTC m=+26.653986099" watchObservedRunningTime="2026-07-16 21:39:20.854656258 +0000 UTC m=+26.655542910" Jul 16 21:39:20.997749 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.997683 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:39:20.998259 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:20.998241 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:39:21.669291 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:21.669256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:21.669475 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:21.669426 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:21.833711 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:21.833685 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:39:21.834143 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:21.833961 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rdcql" Jul 16 21:39:22.669789 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:22.669757 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:22.669955 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:22.669899 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:22.670175 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:22.670052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:22.670175 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:22.670141 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:22.839705 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:22.839669 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" event={"ID":"22066cc8-4543-4e00-ae54-327ba800f77f","Type":"ContainerStarted","Data":"dd95bb481f6d8a1dcdd9c51233c500628bda9307e06f408b1d745323f2e070b7"} Jul 16 21:39:22.840641 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:22.839940 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:39:22.856577 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:22.856534 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:39:22.868742 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:22.868706 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" podStartSLOduration=7.580776939 podStartE2EDuration="27.868693119s" totalImagesPullingTime="20.28791618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:22.867719572 +0000 UTC m=+28.668606223" watchObservedRunningTime="2026-07-16 21:39:22.868693119 +0000 UTC m=+28.669579770" Jul 16 21:39:23.669288 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:23.669064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:23.669439 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:23.669338 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:23.843089 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:23.843060 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 21:39:23.843503 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:23.843224 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:39:23.865222 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:23.865192 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:39:24.087806 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.087769 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dclch"] Jul 16 21:39:24.087951 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.087908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:24.088044 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:24.088025 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:24.088333 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.088315 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4cnbb"] Jul 16 21:39:24.088427 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.088410 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:24.088574 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:24.088510 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:24.089822 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.089800 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddq22"] Jul 16 21:39:24.089922 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.089887 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:24.090035 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:24.089980 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:24.845255 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:24.845186 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 21:39:25.021328 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:25.021302 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:39:25.669636 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:25.669605 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:25.669636 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:25.669623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:25.669867 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:25.669605 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:25.669867 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:25.669708 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:25.669867 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:25.669771 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:25.669867 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:25.669837 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:25.848810 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:25.848769 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7fc97df-84c8-45e9-af2f-1099dc537363" containerID="229bb53e1b696ebf175ec5d942a342fe00a144342987e310f07c729ee2a4197a" exitCode=0 Jul 16 21:39:25.849241 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:25.848868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerDied","Data":"229bb53e1b696ebf175ec5d942a342fe00a144342987e310f07c729ee2a4197a"} Jul 16 21:39:26.852599 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:26.852396 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7fc97df-84c8-45e9-af2f-1099dc537363" containerID="e89098f9113794f228901268da9d6bde5be3c676d4c4474e5620af73fa9bf5d2" exitCode=0 Jul 16 21:39:26.852968 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:26.852484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerDied","Data":"e89098f9113794f228901268da9d6bde5be3c676d4c4474e5620af73fa9bf5d2"} Jul 16 21:39:27.669178 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:27.669158 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:27.669264 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:27.669226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:27.669264 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:27.669238 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:27.669331 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:27.669316 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:27.669396 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:27.669379 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:27.669457 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:27.669439 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:27.857981 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:27.857902 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7fc97df-84c8-45e9-af2f-1099dc537363" containerID="155288e124a497a6f7c8b082fa09eef10b0c0d658b6dde0acc64624c45f9f158" exitCode=0 Jul 16 21:39:27.857981 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:27.857958 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerDied","Data":"155288e124a497a6f7c8b082fa09eef10b0c0d658b6dde0acc64624c45f9f158"} Jul 16 21:39:28.380183 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:28.380151 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:28.380362 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:28.380243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:28.380362 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.380322 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:28.380362 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.380335 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:28.380519 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.380394 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs podName:63a9d655-6a74-4917-ab90-75eec7179b71 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:00.380374859 +0000 UTC m=+66.181261503 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs") pod "network-metrics-daemon-dclch" (UID: "63a9d655-6a74-4917-ab90-75eec7179b71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:39:28.380519 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.380412 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret podName:853ab9b3-3a12-4858-a741-abb04892dd5b nodeName:}" failed. No retries permitted until 2026-07-16 21:40:00.380402379 +0000 UTC m=+66.181289013 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret") pod "global-pull-secret-syncer-ddq22" (UID: "853ab9b3-3a12-4858-a741-abb04892dd5b") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:39:28.480930 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:28.480900 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:28.481072 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.481059 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:39:28.481111 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.481075 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:39:28.481111 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.481084 2578 projected.go:196] Error preparing data for projected volume kube-api-access-jfkrj for pod openshift-network-diagnostics/network-check-target-4cnbb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:28.481169 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:28.481133 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj podName:0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:00.481114562 +0000 UTC m=+66.282001191 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfkrj" (UniqueName: "kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj") pod "network-check-target-4cnbb" (UID: "0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:39:29.669649 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:29.669612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:29.669649 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:29.669626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:29.670109 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:29.669620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:29.670109 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:29.669730 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cnbb" podUID="0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2" Jul 16 21:39:29.670109 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:29.669820 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dclch" podUID="63a9d655-6a74-4917-ab90-75eec7179b71" Jul 16 21:39:29.670109 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:29.669906 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddq22" podUID="853ab9b3-3a12-4858-a741-abb04892dd5b" Jul 16 21:39:29.993831 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:29.993765 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-44.ec2.internal" event="NodeReady" Jul 16 21:39:29.993943 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:29.993866 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 21:39:30.046113 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.046081 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s4d5p"] Jul 16 21:39:30.048768 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.048752 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84bdd9894d-ftm6x"] Jul 16 21:39:30.048898 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.048883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.051325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.051304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.051763 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.051740 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.051875 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.051856 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.051919 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.051911 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 21:39:30.052106 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.052094 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-47jtk\"" Jul 16 21:39:30.052202 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.052188 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 21:39:30.053984 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.053959 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 21:39:30.054052 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.054015 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 21:39:30.054353 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.054335 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcgfg\"" Jul 16 21:39:30.054913 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.054895 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 21:39:30.063869 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.063848 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cv7nc"] Jul 16 21:39:30.069736 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.066808 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 21:39:30.069736 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.067432 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s4d5p"] Jul 16 21:39:30.069736 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.067472 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-62gc6"] Jul 16 21:39:30.069942 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.069864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.071288 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.071263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 21:39:30.071693 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.071674 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.071977 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.071956 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw"] Jul 16 21:39:30.074700 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.074680 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n"] Jul 16 21:39:30.074824 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.074804 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" Jul 16 21:39:30.075457 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075341 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 21:39:30.075457 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075366 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 21:39:30.076168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075631 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 21:39:30.076168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075651 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r6htx\"" Jul 16 21:39:30.076168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075703 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2qh2q\"" Jul 16 21:39:30.076168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075881 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.076168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.075926 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.076168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.076059 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 21:39:30.077748 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.077719 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4"] Jul 16 21:39:30.077857 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.077833 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.079546 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.079372 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 21:39:30.080203 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080178 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.080285 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080272 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9f6px\"" Jul 16 21:39:30.080368 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080338 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 21:39:30.080423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080343 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.080423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080399 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gvkdk\"" Jul 16 21:39:30.080656 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080642 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6575ff56fb-h4b7b"] Jul 16 21:39:30.080762 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080747 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.080838 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080823 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.080901 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.080862 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.083412 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.083377 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67"] Jul 16 21:39:30.083510 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.083491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.083593 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.083538 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 21:39:30.083976 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.083957 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qgth9\"" Jul 16 21:39:30.084056 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.083972 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.084056 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.083969 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.084056 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.084003 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 21:39:30.086591 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.086529 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw"] Jul 16 21:39:30.086693 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.086608 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 21:39:30.086693 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.086643 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 21:39:30.086805 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.086701 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.087092 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.087071 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 21:39:30.087748 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.087728 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.087842 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.087750 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.087842 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.087751 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 21:39:30.087931 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.087855 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4xjg2\"" Jul 16 21:39:30.090840 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.090819 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84bdd9894d-ftm6x"] Jul 16 21:39:30.090933 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.090856 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-62gc6"] Jul 16 21:39:30.090933 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.090930 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.092343 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.092320 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.092428 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.092398 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 21:39:30.092760 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.092741 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 21:39:30.092853 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.092797 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rstw5\"" Jul 16 21:39:30.092990 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.092969 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.094529 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.094513 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cv7nc"] Jul 16 21:39:30.095323 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.095307 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw"] Jul 16 21:39:30.096878 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.096859 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4"] Jul 16 21:39:30.096975 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.096965 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 21:39:30.097665 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.097647 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n"] Jul 16 21:39:30.098000 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.097984 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.101139 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.098520 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2c2f5\"" Jul 16 21:39:30.101235 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.098577 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.101518 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.098721 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 21:39:30.101685 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.098930 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6575ff56fb-h4b7b"] Jul 16 21:39:30.101879 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.101862 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-slv9x"] Jul 16 21:39:30.107433 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.107413 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv"] Jul 16 21:39:30.109113 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.109091 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.110886 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.110867 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx"] Jul 16 21:39:30.111128 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.110932 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.112378 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.112361 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.113051 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.113034 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 21:39:30.113239 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.113184 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wdq25\"" Jul 16 21:39:30.113315 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.113277 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.114307 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.114276 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 21:39:30.114656 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.114636 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 21:39:30.114751 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.114667 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-p6tqr\"" Jul 16 21:39:30.117780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.117758 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67"] Jul 16 21:39:30.117853 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.117796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw"] Jul 16 21:39:30.117853 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.117816 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx"] Jul 16 21:39:30.117853 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.117821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" Jul 16 21:39:30.117853 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.117834 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv"] Jul 16 21:39:30.117853 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.117851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-slv9x"] Jul 16 21:39:30.120258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.120240 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 21:39:30.120446 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.120432 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-zvftp\"" Jul 16 21:39:30.120735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.120716 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 21:39:30.191773 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.191773 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fe2c709-066d-4f62-a8f6-b191620dcf80-config\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.191948 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-installation-pull-secrets\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.191948 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59fe218f-3f52-4ddb-b052-762f1e9ddef8-cert\") pod \"ingress-canary-slv9x\" (UID: \"59fe218f-3f52-4ddb-b052-762f1e9ddef8\") " pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.191948 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b048d364-094e-443b-a259-7d836250a940-trusted-ca\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.191948 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpwvb\" (UniqueName: \"kubernetes.io/projected/b048d364-094e-443b-a259-7d836250a940-kube-api-access-lpwvb\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.191948 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e020bb33-9484-4dcc-b073-2eb4e9e90666-ca-trust-extracted\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.191948 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-bound-sa-token\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6594bdb-a475-4973-8068-f6bf5ddd341a-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.191978 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-certificates\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/192d89ab-ec22-4974-a8b2-01f6eb9778fd-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lz6w\" (UniqueName: \"kubernetes.io/projected/f6571ef5-21df-43b3-b213-dd81d521d969-kube-api-access-9lz6w\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cz45\" (UniqueName: \"kubernetes.io/projected/192d89ab-ec22-4974-a8b2-01f6eb9778fd-kube-api-access-5cz45\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b078fc92-aee9-4bf5-a3b6-bce92c892808-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-default-certificate\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95tv\" (UniqueName: \"kubernetes.io/projected/59fe218f-3f52-4ddb-b052-762f1e9ddef8-kube-api-access-n95tv\") pod \"ingress-canary-slv9x\" (UID: \"59fe218f-3f52-4ddb-b052-762f1e9ddef8\") " pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw8s8\" (UniqueName: \"kubernetes.io/projected/b078fc92-aee9-4bf5-a3b6-bce92c892808-kube-api-access-hw8s8\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.192246 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgm8q\" (UniqueName: \"kubernetes.io/projected/1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554-kube-api-access-vgm8q\") pod \"network-check-source-7bc5b6bf85-tk5mx\" (UID: \"1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b078fc92-aee9-4bf5-a3b6-bce92c892808-tmp\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fe2c709-066d-4f62-a8f6-b191620dcf80-serving-cert\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtfxd\" (UniqueName: \"kubernetes.io/projected/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-kube-api-access-wtfxd\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-555gs\" (UniqueName: \"kubernetes.io/projected/3fe2c709-066d-4f62-a8f6-b191620dcf80-kube-api-access-555gs\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7d44\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-kube-api-access-d7d44\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6594bdb-a475-4973-8068-f6bf5ddd341a-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192604 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxm8b\" (UniqueName: \"kubernetes.io/projected/b9326bb8-a637-44ec-b922-ab4f9f87a429-kube-api-access-gxm8b\") pod \"volume-data-source-validator-78cf4c6f6-2kpcw\" (UID: \"b9326bb8-a637-44ec-b922-ab4f9f87a429\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-trusted-ca\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b078fc92-aee9-4bf5-a3b6-bce92c892808-serving-cert\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-image-registry-private-configuration\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6571ef5-21df-43b3-b213-dd81d521d969-config-volume\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.192780 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-stats-auth\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzz68\" (UniqueName: \"kubernetes.io/projected/c6594bdb-a475-4973-8068-f6bf5ddd341a-kube-api-access-kzz68\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6571ef5-21df-43b3-b213-dd81d521d969-tmp-dir\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w42lh\" (UniqueName: \"kubernetes.io/projected/88a3cdc3-9038-4524-8fce-d25d69418bc6-kube-api-access-w42lh\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b048d364-094e-443b-a259-7d836250a940-config\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b048d364-094e-443b-a259-7d836250a940-serving-cert\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.192991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b078fc92-aee9-4bf5-a3b6-bce92c892808-snapshots\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.193372 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.193016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b078fc92-aee9-4bf5-a3b6-bce92c892808-service-ca-bundle\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.293862 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.293830 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e020bb33-9484-4dcc-b073-2eb4e9e90666-ca-trust-extracted\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.294020 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.293873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-bound-sa-token\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.294020 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.293899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6594bdb-a475-4973-8068-f6bf5ddd341a-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.294020 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.293947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.294020 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.293992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-certificates\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.294020 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/192d89ab-ec22-4974-a8b2-01f6eb9778fd-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lz6w\" (UniqueName: \"kubernetes.io/projected/f6571ef5-21df-43b3-b213-dd81d521d969-kube-api-access-9lz6w\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cz45\" (UniqueName: \"kubernetes.io/projected/192d89ab-ec22-4974-a8b2-01f6eb9778fd-kube-api-access-5cz45\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b078fc92-aee9-4bf5-a3b6-bce92c892808-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-default-certificate\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95tv\" (UniqueName: \"kubernetes.io/projected/59fe218f-3f52-4ddb-b052-762f1e9ddef8-kube-api-access-n95tv\") pod \"ingress-canary-slv9x\" (UID: \"59fe218f-3f52-4ddb-b052-762f1e9ddef8\") " pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw8s8\" (UniqueName: \"kubernetes.io/projected/b078fc92-aee9-4bf5-a3b6-bce92c892808-kube-api-access-hw8s8\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.294160 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.294219 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.294232 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert podName:293491f1-a543-4ae5-a8f9-ccdf2cd8957b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.79421187 +0000 UTC m=+36.595098500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-t25mv" (UID: "293491f1-a543-4ae5-a8f9-ccdf2cd8957b") : secret "networking-console-plugin-cert" not found Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.294258 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.294266 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.794249463 +0000 UTC m=+36.595136092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : secret "router-metrics-certs-default" not found Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.294293 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm8q\" (UniqueName: \"kubernetes.io/projected/1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554-kube-api-access-vgm8q\") pod \"network-check-source-7bc5b6bf85-tk5mx\" (UID: \"1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.294330 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls podName:88a3cdc3-9038-4524-8fce-d25d69418bc6 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.79431471 +0000 UTC m=+36.595201341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsx9n" (UID: "88a3cdc3-9038-4524-8fce-d25d69418bc6") : secret "samples-operator-tls" not found Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b078fc92-aee9-4bf5-a3b6-bce92c892808-tmp\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fe2c709-066d-4f62-a8f6-b191620dcf80-serving-cert\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e020bb33-9484-4dcc-b073-2eb4e9e90666-ca-trust-extracted\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtfxd\" (UniqueName: \"kubernetes.io/projected/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-kube-api-access-wtfxd\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-555gs\" (UniqueName: \"kubernetes.io/projected/3fe2c709-066d-4f62-a8f6-b191620dcf80-kube-api-access-555gs\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7d44\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-kube-api-access-d7d44\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6594bdb-a475-4973-8068-f6bf5ddd341a-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxm8b\" (UniqueName: \"kubernetes.io/projected/b9326bb8-a637-44ec-b922-ab4f9f87a429-kube-api-access-gxm8b\") pod \"volume-data-source-validator-78cf4c6f6-2kpcw\" (UID: \"b9326bb8-a637-44ec-b922-ab4f9f87a429\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-trusted-ca\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b078fc92-aee9-4bf5-a3b6-bce92c892808-serving-cert\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.294888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294784 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-certificates\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-image-registry-private-configuration\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6571ef5-21df-43b3-b213-dd81d521d969-config-volume\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-stats-auth\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzz68\" (UniqueName: \"kubernetes.io/projected/c6594bdb-a475-4973-8068-f6bf5ddd341a-kube-api-access-kzz68\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6571ef5-21df-43b3-b213-dd81d521d969-tmp-dir\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w42lh\" (UniqueName: \"kubernetes.io/projected/88a3cdc3-9038-4524-8fce-d25d69418bc6-kube-api-access-w42lh\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b048d364-094e-443b-a259-7d836250a940-config\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.294996 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b048d364-094e-443b-a259-7d836250a940-serving-cert\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b078fc92-aee9-4bf5-a3b6-bce92c892808-snapshots\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b078fc92-aee9-4bf5-a3b6-bce92c892808-service-ca-bundle\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fe2c709-066d-4f62-a8f6-b191620dcf80-config\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.295619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-installation-pull-secrets\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59fe218f-3f52-4ddb-b052-762f1e9ddef8-cert\") pod \"ingress-canary-slv9x\" (UID: \"59fe218f-3f52-4ddb-b052-762f1e9ddef8\") " pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b048d364-094e-443b-a259-7d836250a940-trusted-ca\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpwvb\" (UniqueName: \"kubernetes.io/projected/b048d364-094e-443b-a259-7d836250a940-kube-api-access-lpwvb\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295289 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/192d89ab-ec22-4974-a8b2-01f6eb9778fd-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.295344 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.295394 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls podName:f6571ef5-21df-43b3-b213-dd81d521d969 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.795379552 +0000 UTC m=+36.596266183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls") pod "dns-default-cv7nc" (UID: "f6571ef5-21df-43b3-b213-dd81d521d969") : secret "dns-default-metrics-tls" not found Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295704 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6594bdb-a475-4973-8068-f6bf5ddd341a-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295960 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6571ef5-21df-43b3-b213-dd81d521d969-tmp-dir\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295964 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b078fc92-aee9-4bf5-a3b6-bce92c892808-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.296225 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.295956 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b078fc92-aee9-4bf5-a3b6-bce92c892808-tmp\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.296281 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.296330 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls podName:192d89ab-ec22-4974-a8b2-01f6eb9778fd nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.796315526 +0000 UTC m=+36.597202169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xzmgw" (UID: "192d89ab-ec22-4974-a8b2-01f6eb9778fd") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.296349 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-trusted-ca\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.296453 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.796436417 +0000 UTC m=+36.597323066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : configmap references non-existent config key: service-ca.crt Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.296584 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.296597 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84bdd9894d-ftm6x: secret "image-registry-tls" not found Jul 16 21:39:30.296724 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.296647 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls podName:e020bb33-9484-4dcc-b073-2eb4e9e90666 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:30.796632468 +0000 UTC m=+36.597519098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls") pod "image-registry-84bdd9894d-ftm6x" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666") : secret "image-registry-tls" not found Jul 16 21:39:30.297072 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.296807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6571ef5-21df-43b3-b213-dd81d521d969-config-volume\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.297072 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.296976 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b048d364-094e-443b-a259-7d836250a940-config\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.297168 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.297129 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b078fc92-aee9-4bf5-a3b6-bce92c892808-snapshots\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.297685 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.297588 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b078fc92-aee9-4bf5-a3b6-bce92c892808-service-ca-bundle\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.297864 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.297840 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fe2c709-066d-4f62-a8f6-b191620dcf80-config\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.299451 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.299417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b048d364-094e-443b-a259-7d836250a940-trusted-ca\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.301062 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.301041 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b078fc92-aee9-4bf5-a3b6-bce92c892808-serving-cert\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.301154 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.301060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b048d364-094e-443b-a259-7d836250a940-serving-cert\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.301407 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.301359 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6594bdb-a475-4973-8068-f6bf5ddd341a-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.301607 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.301541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-image-registry-private-configuration\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.301691 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.301670 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59fe218f-3f52-4ddb-b052-762f1e9ddef8-cert\") pod \"ingress-canary-slv9x\" (UID: \"59fe218f-3f52-4ddb-b052-762f1e9ddef8\") " pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.302485 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.302433 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-default-certificate\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.302587 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.302502 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-installation-pull-secrets\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.302587 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.302510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3fe2c709-066d-4f62-a8f6-b191620dcf80-serving-cert\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.303079 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.303042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-stats-auth\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.303870 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.303824 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgm8q\" (UniqueName: \"kubernetes.io/projected/1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554-kube-api-access-vgm8q\") pod \"network-check-source-7bc5b6bf85-tk5mx\" (UID: \"1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" Jul 16 21:39:30.308505 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.307648 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpwvb\" (UniqueName: \"kubernetes.io/projected/b048d364-094e-443b-a259-7d836250a940-kube-api-access-lpwvb\") pod \"console-operator-565b7b569d-s4d5p\" (UID: \"b048d364-094e-443b-a259-7d836250a940\") " pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.308505 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.307901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-555gs\" (UniqueName: \"kubernetes.io/projected/3fe2c709-066d-4f62-a8f6-b191620dcf80-kube-api-access-555gs\") pod \"service-ca-operator-5bcbc94c78-2lsl4\" (UID: \"3fe2c709-066d-4f62-a8f6-b191620dcf80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.308505 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.308453 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtfxd\" (UniqueName: \"kubernetes.io/projected/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-kube-api-access-wtfxd\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.309750 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.309346 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-bound-sa-token\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.309750 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.309661 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7d44\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-kube-api-access-d7d44\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.310096 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.310073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzz68\" (UniqueName: \"kubernetes.io/projected/c6594bdb-a475-4973-8068-f6bf5ddd341a-kube-api-access-kzz68\") pod \"kube-storage-version-migrator-operator-7658d9b749-5mx67\" (UID: \"c6594bdb-a475-4973-8068-f6bf5ddd341a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.310646 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.310627 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxm8b\" (UniqueName: \"kubernetes.io/projected/b9326bb8-a637-44ec-b922-ab4f9f87a429-kube-api-access-gxm8b\") pod \"volume-data-source-validator-78cf4c6f6-2kpcw\" (UID: \"b9326bb8-a637-44ec-b922-ab4f9f87a429\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" Jul 16 21:39:30.312668 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.312618 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw8s8\" (UniqueName: \"kubernetes.io/projected/b078fc92-aee9-4bf5-a3b6-bce92c892808-kube-api-access-hw8s8\") pod \"insights-operator-555d9d7885-62gc6\" (UID: \"b078fc92-aee9-4bf5-a3b6-bce92c892808\") " pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.319243 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.319218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lz6w\" (UniqueName: \"kubernetes.io/projected/f6571ef5-21df-43b3-b213-dd81d521d969-kube-api-access-9lz6w\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.319583 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.319326 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cz45\" (UniqueName: \"kubernetes.io/projected/192d89ab-ec22-4974-a8b2-01f6eb9778fd-kube-api-access-5cz45\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.321062 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.321037 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95tv\" (UniqueName: \"kubernetes.io/projected/59fe218f-3f52-4ddb-b052-762f1e9ddef8-kube-api-access-n95tv\") pod \"ingress-canary-slv9x\" (UID: \"59fe218f-3f52-4ddb-b052-762f1e9ddef8\") " pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.322049 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.322028 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w42lh\" (UniqueName: \"kubernetes.io/projected/88a3cdc3-9038-4524-8fce-d25d69418bc6-kube-api-access-w42lh\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.359311 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.359284 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:30.391384 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.391332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-62gc6" Jul 16 21:39:30.399958 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.399933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" Jul 16 21:39:30.421432 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.421407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" Jul 16 21:39:30.459580 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.446014 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" Jul 16 21:39:30.459580 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.452647 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-slv9x" Jul 16 21:39:30.471870 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.469810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" Jul 16 21:39:30.579758 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.578525 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s4d5p"] Jul 16 21:39:30.627539 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.626937 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-62gc6"] Jul 16 21:39:30.674042 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.673987 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw"] Jul 16 21:39:30.676142 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:39:30.676108 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9326bb8_a637_44ec_b922_ab4f9f87a429.slice/crio-c80f59bed30ee0875368663fce88c674938613adda6359b805528dfc3ed1fcb6 WatchSource:0}: Error finding container c80f59bed30ee0875368663fce88c674938613adda6359b805528dfc3ed1fcb6: Status 404 returned error can't find the container with id c80f59bed30ee0875368663fce88c674938613adda6359b805528dfc3ed1fcb6 Jul 16 21:39:30.685886 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.684246 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4"] Jul 16 21:39:30.689121 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:39:30.688871 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe2c709_066d_4f62_a8f6_b191620dcf80.slice/crio-2a5cfe5b785e5ab17ee76d262774793315886df36f1ec4a75a4f7589bf39d7d1 WatchSource:0}: Error finding container 2a5cfe5b785e5ab17ee76d262774793315886df36f1ec4a75a4f7589bf39d7d1: Status 404 returned error can't find the container with id 2a5cfe5b785e5ab17ee76d262774793315886df36f1ec4a75a4f7589bf39d7d1 Jul 16 21:39:30.701896 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.701867 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-slv9x"] Jul 16 21:39:30.705130 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:39:30.705100 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59fe218f_3f52_4ddb_b052_762f1e9ddef8.slice/crio-ebe666c1eb7187261dd6adf184a7831fcb343a560b62b8f173a2cefd0176b7d3 WatchSource:0}: Error finding container ebe666c1eb7187261dd6adf184a7831fcb343a560b62b8f173a2cefd0176b7d3: Status 404 returned error can't find the container with id ebe666c1eb7187261dd6adf184a7831fcb343a560b62b8f173a2cefd0176b7d3 Jul 16 21:39:30.715298 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.715232 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67"] Jul 16 21:39:30.719086 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:39:30.719059 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6594bdb_a475_4973_8068_f6bf5ddd341a.slice/crio-fe513abebd5501fe944ebe0920dc0c1b01f59874a2266272dd05144e4833ad1c WatchSource:0}: Error finding container fe513abebd5501fe944ebe0920dc0c1b01f59874a2266272dd05144e4833ad1c: Status 404 returned error can't find the container with id fe513abebd5501fe944ebe0920dc0c1b01f59874a2266272dd05144e4833ad1c Jul 16 21:39:30.739181 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.739157 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx"] Jul 16 21:39:30.742571 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:39:30.742520 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc7a4e6_aecc_4231_9b2a_a9d6d5b7f554.slice/crio-a159bc4263ff42499e1a06bdb247abcddf40b21542091f79f94f9226de4bc435 WatchSource:0}: Error finding container a159bc4263ff42499e1a06bdb247abcddf40b21542091f79f94f9226de4bc435: Status 404 returned error can't find the container with id a159bc4263ff42499e1a06bdb247abcddf40b21542091f79f94f9226de4bc435 Jul 16 21:39:30.800147 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:30.800147 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800247 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.800299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800304 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 21:39:30.800314 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800316 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls podName:192d89ab-ec22-4974-a8b2-01f6eb9778fd nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800295804 +0000 UTC m=+37.601182435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xzmgw" (UID: "192d89ab-ec22-4974-a8b2-01f6eb9778fd") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800250 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800350 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84bdd9894d-ftm6x: secret "image-registry-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800357 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert podName:293491f1-a543-4ae5-a8f9-ccdf2cd8957b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800340573 +0000 UTC m=+37.601227207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-t25mv" (UID: "293491f1-a543-4ae5-a8f9-ccdf2cd8957b") : secret "networking-console-plugin-cert" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800380 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls podName:e020bb33-9484-4dcc-b073-2eb4e9e90666 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800369162 +0000 UTC m=+37.601255791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls") pod "image-registry-84bdd9894d-ftm6x" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666") : secret "image-registry-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800399 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800388544 +0000 UTC m=+37.601275174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : configmap references non-existent config key: service-ca.crt Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800425 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800453 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800443784 +0000 UTC m=+37.601330415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : secret "router-metrics-certs-default" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800463 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800497 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls podName:f6571ef5-21df-43b3-b213-dd81d521d969 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800486153 +0000 UTC m=+37.601372783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls") pod "dns-default-cv7nc" (UID: "f6571ef5-21df-43b3-b213-dd81d521d969") : secret "dns-default-metrics-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800501 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:39:30.800753 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:30.800528 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls podName:88a3cdc3-9038-4524-8fce-d25d69418bc6 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:31.800519408 +0000 UTC m=+37.601406037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsx9n" (UID: "88a3cdc3-9038-4524-8fce-d25d69418bc6") : secret "samples-operator-tls" not found Jul 16 21:39:30.867450 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.867418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" event={"ID":"c6594bdb-a475-4973-8068-f6bf5ddd341a","Type":"ContainerStarted","Data":"fe513abebd5501fe944ebe0920dc0c1b01f59874a2266272dd05144e4833ad1c"} Jul 16 21:39:30.868455 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.868436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" event={"ID":"b9326bb8-a637-44ec-b922-ab4f9f87a429","Type":"ContainerStarted","Data":"c80f59bed30ee0875368663fce88c674938613adda6359b805528dfc3ed1fcb6"} Jul 16 21:39:30.869470 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.869453 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-slv9x" event={"ID":"59fe218f-3f52-4ddb-b052-762f1e9ddef8","Type":"ContainerStarted","Data":"ebe666c1eb7187261dd6adf184a7831fcb343a560b62b8f173a2cefd0176b7d3"} Jul 16 21:39:30.870459 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.870436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" event={"ID":"b048d364-094e-443b-a259-7d836250a940","Type":"ContainerStarted","Data":"81370c7f5d84a0a99706480bd4734705df34eacd5cb0ab5d2f2c33f5c6a95594"} Jul 16 21:39:30.871361 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.871342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" event={"ID":"1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554","Type":"ContainerStarted","Data":"a159bc4263ff42499e1a06bdb247abcddf40b21542091f79f94f9226de4bc435"} Jul 16 21:39:30.872234 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.872217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" event={"ID":"3fe2c709-066d-4f62-a8f6-b191620dcf80","Type":"ContainerStarted","Data":"2a5cfe5b785e5ab17ee76d262774793315886df36f1ec4a75a4f7589bf39d7d1"} Jul 16 21:39:30.873173 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:30.873157 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-62gc6" event={"ID":"b078fc92-aee9-4bf5-a3b6-bce92c892808","Type":"ContainerStarted","Data":"5515ad71abf7cb874f9b4a7697ed7993427b3398c30f277621efac77008f13bb"} Jul 16 21:39:31.669388 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.669326 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:39:31.669673 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.669655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:39:31.670213 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.670194 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:39:31.672392 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.672179 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 21:39:31.673582 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.673132 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bjvwm\"" Jul 16 21:39:31.673582 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.673359 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cd5nk\"" Jul 16 21:39:31.676580 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.676543 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 21:39:31.809409 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:31.809665 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:31.809665 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:31.809665 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:31.809665 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:31.809665 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:31.809926 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:31.809684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:31.809926 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.809830 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.809811739 +0000 UTC m=+39.610698386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : configmap references non-existent config key: service-ca.crt Jul 16 21:39:31.810280 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810259 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 21:39:31.810355 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810317 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert podName:293491f1-a543-4ae5-a8f9-ccdf2cd8957b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.810301448 +0000 UTC m=+39.611188077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-t25mv" (UID: "293491f1-a543-4ae5-a8f9-ccdf2cd8957b") : secret "networking-console-plugin-cert" not found Jul 16 21:39:31.810422 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810381 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:39:31.810422 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810416 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls podName:88a3cdc3-9038-4524-8fce-d25d69418bc6 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.810404738 +0000 UTC m=+39.611291370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsx9n" (UID: "88a3cdc3-9038-4524-8fce-d25d69418bc6") : secret "samples-operator-tls" not found Jul 16 21:39:31.810520 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810474 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:39:31.810520 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810504 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.810493994 +0000 UTC m=+39.611380630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : secret "router-metrics-certs-default" not found Jul 16 21:39:31.810695 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810672 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:31.810750 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810717 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls podName:192d89ab-ec22-4974-a8b2-01f6eb9778fd nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.810703644 +0000 UTC m=+39.611590274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xzmgw" (UID: "192d89ab-ec22-4974-a8b2-01f6eb9778fd") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:31.810815 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810778 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:39:31.810815 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810810 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls podName:f6571ef5-21df-43b3-b213-dd81d521d969 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.810798212 +0000 UTC m=+39.611684846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls") pod "dns-default-cv7nc" (UID: "f6571ef5-21df-43b3-b213-dd81d521d969") : secret "dns-default-metrics-tls" not found Jul 16 21:39:31.810920 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810871 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:39:31.810920 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810882 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84bdd9894d-ftm6x: secret "image-registry-tls" not found Jul 16 21:39:31.810920 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:31.810914 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls podName:e020bb33-9484-4dcc-b073-2eb4e9e90666 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:33.810903168 +0000 UTC m=+39.611789800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls") pod "image-registry-84bdd9894d-ftm6x" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666") : secret "image-registry-tls" not found Jul 16 21:39:33.830875 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.830842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.830896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.830924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.830988 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.831003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831038 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831066 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert podName:293491f1-a543-4ae5-a8f9-ccdf2cd8957b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.831042814 +0000 UTC m=+43.631929454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-t25mv" (UID: "293491f1-a543-4ae5-a8f9-ccdf2cd8957b") : secret "networking-console-plugin-cert" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831096 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls podName:88a3cdc3-9038-4524-8fce-d25d69418bc6 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.83108085 +0000 UTC m=+43.631967478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsx9n" (UID: "88a3cdc3-9038-4524-8fce-d25d69418bc6") : secret "samples-operator-tls" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.831095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831116 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.831147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831169 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831188 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.831153587 +0000 UTC m=+43.632040216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : secret "router-metrics-certs-default" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831204 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831214 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84bdd9894d-ftm6x: secret "image-registry-tls" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831207 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls podName:f6571ef5-21df-43b3-b213-dd81d521d969 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.831201299 +0000 UTC m=+43.632087928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls") pod "dns-default-cv7nc" (UID: "f6571ef5-21df-43b3-b213-dd81d521d969") : secret "dns-default-metrics-tls" not found Jul 16 21:39:33.831325 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831249 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls podName:e020bb33-9484-4dcc-b073-2eb4e9e90666 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.831237949 +0000 UTC m=+43.632124578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls") pod "image-registry-84bdd9894d-ftm6x" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666") : secret "image-registry-tls" not found Jul 16 21:39:33.831847 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831292 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:33.831847 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:33.831297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:33.831847 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831344 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls podName:192d89ab-ec22-4974-a8b2-01f6eb9778fd nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.831319472 +0000 UTC m=+43.632206106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xzmgw" (UID: "192d89ab-ec22-4974-a8b2-01f6eb9778fd") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:33.831847 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:33.831368 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:37.831361552 +0000 UTC m=+43.632248180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : configmap references non-existent config key: service-ca.crt Jul 16 21:39:37.868137 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868249 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868274 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84bdd9894d-ftm6x: secret "image-registry-tls" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868322 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls podName:e020bb33-9484-4dcc-b073-2eb4e9e90666 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.868303657 +0000 UTC m=+51.669190286 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls") pod "image-registry-84bdd9894d-ftm6x" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666") : secret "image-registry-tls" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868349 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868369 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868377 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868428 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.868394274 +0000 UTC m=+51.669280908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : configmap references non-existent config key: service-ca.crt Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868457 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert podName:293491f1-a543-4ae5-a8f9-ccdf2cd8957b nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.868446707 +0000 UTC m=+51.669333347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-t25mv" (UID: "293491f1-a543-4ae5-a8f9-ccdf2cd8957b") : secret "networking-console-plugin-cert" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868474 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls podName:88a3cdc3-9038-4524-8fce-d25d69418bc6 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.868465504 +0000 UTC m=+51.669352133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsx9n" (UID: "88a3cdc3-9038-4524-8fce-d25d69418bc6") : secret "samples-operator-tls" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868500 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.868490055 +0000 UTC m=+51.669376687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : secret "router-metrics-certs-default" not found Jul 16 21:39:37.868688 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:37.869365 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:37.868660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:37.869365 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868755 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:39:37.869365 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868773 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:37.869365 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868795 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls podName:f6571ef5-21df-43b3-b213-dd81d521d969 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.86878307 +0000 UTC m=+51.669669719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls") pod "dns-default-cv7nc" (UID: "f6571ef5-21df-43b3-b213-dd81d521d969") : secret "dns-default-metrics-tls" not found Jul 16 21:39:37.869365 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:37.868818 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls podName:192d89ab-ec22-4974-a8b2-01f6eb9778fd nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.868805265 +0000 UTC m=+51.669691912 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xzmgw" (UID: "192d89ab-ec22-4974-a8b2-01f6eb9778fd") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:40.907938 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.907852 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7fc97df-84c8-45e9-af2f-1099dc537363" containerID="24abd31c75d53f5ea9abbcb877f7be6de55f2a1fb77d57db0ac97132b271c77b" exitCode=0 Jul 16 21:39:40.907938 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.907932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerDied","Data":"24abd31c75d53f5ea9abbcb877f7be6de55f2a1fb77d57db0ac97132b271c77b"} Jul 16 21:39:40.910129 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.910083 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" event={"ID":"1fc7a4e6-aecc-4231-9b2a-a9d6d5b7f554","Type":"ContainerStarted","Data":"c947365327afb1fc814b147d6756f48e9983c940c0b20b560ed1fa6971af7a61"} Jul 16 21:39:40.915873 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.915778 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" event={"ID":"3fe2c709-066d-4f62-a8f6-b191620dcf80","Type":"ContainerStarted","Data":"95fc303398cc79518c91e4fe01cd862d346a062a63eea88ea0f2e1cf28e72d53"} Jul 16 21:39:40.919176 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.919147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-62gc6" event={"ID":"b078fc92-aee9-4bf5-a3b6-bce92c892808","Type":"ContainerStarted","Data":"a29540f5c211d1664a4b5544d631e59f6dd1f3ab17c36edfd139105760967cb9"} Jul 16 21:39:40.920683 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.920653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" event={"ID":"c6594bdb-a475-4973-8068-f6bf5ddd341a","Type":"ContainerStarted","Data":"fa477787b1bcddb55fc88381e10cfa5906efa8889b488a791b4a1d93c613c8be"} Jul 16 21:39:40.922154 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.922132 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" event={"ID":"b9326bb8-a637-44ec-b922-ab4f9f87a429","Type":"ContainerStarted","Data":"5826c67f67f397002b52fd80b5ade6fd0c1c43663bedc785e6510c8c36f43dc2"} Jul 16 21:39:40.923578 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.923537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-slv9x" event={"ID":"59fe218f-3f52-4ddb-b052-762f1e9ddef8","Type":"ContainerStarted","Data":"c832cd12d30ef0c1fd9670a90633dad28e89f89dd8bff12ace0fb13194006471"} Jul 16 21:39:40.925036 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.925018 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/0.log" Jul 16 21:39:40.925143 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.925054 2578 generic.go:356] "Generic (PLEG): container finished" podID="b048d364-094e-443b-a259-7d836250a940" containerID="ef5796be5a4fec27d8808a82711bf0101cb5a63247b287c3dc5040b0e0c6882f" exitCode=255 Jul 16 21:39:40.925143 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.925080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" event={"ID":"b048d364-094e-443b-a259-7d836250a940","Type":"ContainerDied","Data":"ef5796be5a4fec27d8808a82711bf0101cb5a63247b287c3dc5040b0e0c6882f"} Jul 16 21:39:40.925340 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.925326 2578 scope.go:117] "RemoveContainer" containerID="ef5796be5a4fec27d8808a82711bf0101cb5a63247b287c3dc5040b0e0c6882f" Jul 16 21:39:40.955508 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.955461 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-62gc6" podStartSLOduration=21.209348008 podStartE2EDuration="30.955443354s" totalImagesPullingTime="9.746095346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:40.955344225 +0000 UTC m=+46.756230877" watchObservedRunningTime="2026-07-16 21:39:40.955443354 +0000 UTC m=+46.756330009" Jul 16 21:39:40.976585 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:40.975065 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" podStartSLOduration=21.3061213 podStartE2EDuration="30.975044348s" totalImagesPullingTime="9.668923048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:40.973803872 +0000 UTC m=+46.774690523" watchObservedRunningTime="2026-07-16 21:39:40.975044348 +0000 UTC m=+46.775931001" Jul 16 21:39:41.010529 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.010461 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" podStartSLOduration=21.305517948 podStartE2EDuration="31.010440137s" totalImagesPullingTime="9.704922189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:40.991319254 +0000 UTC m=+46.792205909" watchObservedRunningTime="2026-07-16 21:39:41.010440137 +0000 UTC m=+46.811326790" Jul 16 21:39:41.025173 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.025125 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-2kpcw" podStartSLOduration=21.313977776 podStartE2EDuration="31.025112173s" totalImagesPullingTime="9.711134397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:41.023975845 +0000 UTC m=+46.824862496" watchObservedRunningTime="2026-07-16 21:39:41.025112173 +0000 UTC m=+46.825998825" Jul 16 21:39:41.046573 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.046502 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-slv9x" podStartSLOduration=1.363647166 podStartE2EDuration="11.046482399s" totalImagesPullingTime="9.682835233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:41.044605057 +0000 UTC m=+46.845491711" watchObservedRunningTime="2026-07-16 21:39:41.046482399 +0000 UTC m=+46.847369053" Jul 16 21:39:41.068327 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.067967 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tk5mx" podStartSLOduration=21.422620843 podStartE2EDuration="31.067946332s" totalImagesPullingTime="9.645325489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:41.064412202 +0000 UTC m=+46.865298877" watchObservedRunningTime="2026-07-16 21:39:41.067946332 +0000 UTC m=+46.868832985" Jul 16 21:39:41.741845 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.741753 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rzkg4"] Jul 16 21:39:41.774255 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.774221 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rzkg4"] Jul 16 21:39:41.774422 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.774328 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:41.776932 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.776906 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 21:39:41.776932 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.776925 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 21:39:41.777212 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.777199 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vpjhj\"" Jul 16 21:39:41.904055 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.904017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e93aebfb-7224-44b2-a908-43b5ef7d7897-data-volume\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:41.904223 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.904070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e93aebfb-7224-44b2-a908-43b5ef7d7897-crio-socket\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:41.904223 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.904144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:41.904302 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.904265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgmxp\" (UniqueName: \"kubernetes.io/projected/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-api-access-jgmxp\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:41.904343 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.904304 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:41.930818 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.930793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:39:41.931250 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.931237 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/0.log" Jul 16 21:39:41.931306 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.931274 2578 generic.go:356] "Generic (PLEG): container finished" podID="b048d364-094e-443b-a259-7d836250a940" containerID="b20aa1d83425ddd4ead4f2afd6c71b6f0b4faff09236d8f184ef4ffa517661a0" exitCode=255 Jul 16 21:39:41.931393 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.931364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" event={"ID":"b048d364-094e-443b-a259-7d836250a940","Type":"ContainerDied","Data":"b20aa1d83425ddd4ead4f2afd6c71b6f0b4faff09236d8f184ef4ffa517661a0"} Jul 16 21:39:41.931514 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.931431 2578 scope.go:117] "RemoveContainer" containerID="ef5796be5a4fec27d8808a82711bf0101cb5a63247b287c3dc5040b0e0c6882f" Jul 16 21:39:41.931683 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.931667 2578 scope.go:117] "RemoveContainer" containerID="b20aa1d83425ddd4ead4f2afd6c71b6f0b4faff09236d8f184ef4ffa517661a0" Jul 16 21:39:41.931870 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:41.931849 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s4d5p_openshift-console-operator(b048d364-094e-443b-a259-7d836250a940)\"" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" podUID="b048d364-094e-443b-a259-7d836250a940" Jul 16 21:39:41.935109 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.935089 2578 generic.go:356] "Generic (PLEG): container finished" podID="a7fc97df-84c8-45e9-af2f-1099dc537363" containerID="d822faac7512af24fac056383668295f5ac646a69ca7fda544f2ccef2914c2fe" exitCode=0 Jul 16 21:39:41.935205 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:41.935175 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerDied","Data":"d822faac7512af24fac056383668295f5ac646a69ca7fda544f2ccef2914c2fe"} Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.005403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e93aebfb-7224-44b2-a908-43b5ef7d7897-data-volume\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.005460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e93aebfb-7224-44b2-a908-43b5ef7d7897-crio-socket\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.005505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.006474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgmxp\" (UniqueName: \"kubernetes.io/projected/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-api-access-jgmxp\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.006532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.006676 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.006732 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:42.506712535 +0000 UTC m=+48.307599164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : secret "insights-runtime-extractor-tls" not found Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.006764 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e93aebfb-7224-44b2-a908-43b5ef7d7897-data-volume\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.006940 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.006984 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:42.506970726 +0000 UTC m=+48.307857376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : configmap "kube-rbac-proxy" not found Jul 16 21:39:42.007276 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.007143 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e93aebfb-7224-44b2-a908-43b5ef7d7897-crio-socket\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.016849 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.016826 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgmxp\" (UniqueName: \"kubernetes.io/projected/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-api-access-jgmxp\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.512203 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.512169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.512344 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.512280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:42.512344 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.512319 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:39:42.512426 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.512344 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:39:42.512426 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.512383 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:43.512368424 +0000 UTC m=+49.313255053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : secret "insights-runtime-extractor-tls" not found Jul 16 21:39:42.512426 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.512399 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:43.512391946 +0000 UTC m=+49.313278574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : configmap "kube-rbac-proxy" not found Jul 16 21:39:42.942742 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.942675 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:39:42.943134 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.943021 2578 scope.go:117] "RemoveContainer" containerID="b20aa1d83425ddd4ead4f2afd6c71b6f0b4faff09236d8f184ef4ffa517661a0" Jul 16 21:39:42.943197 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:42.943161 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s4d5p_openshift-console-operator(b048d364-094e-443b-a259-7d836250a940)\"" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" podUID="b048d364-094e-443b-a259-7d836250a940" Jul 16 21:39:42.946032 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.946009 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sb58x" event={"ID":"a7fc97df-84c8-45e9-af2f-1099dc537363","Type":"ContainerStarted","Data":"44cd041df70c73e2e5475480c4969bbec1a712f5dcad7050e2f999c76025557a"} Jul 16 21:39:42.981019 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:42.980972 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sb58x" podStartSLOduration=7.587270652 podStartE2EDuration="47.980959364s" totalImagesPullingTime="40.393688712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:39:42.979296539 +0000 UTC m=+48.780183190" watchObservedRunningTime="2026-07-16 21:39:42.980959364 +0000 UTC m=+48.781846014" Jul 16 21:39:43.520358 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:43.520317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:43.520507 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:43.520434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:43.520507 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:43.520454 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:39:43.520612 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:43.520515 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.520498648 +0000 UTC m=+51.321385278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : configmap "kube-rbac-proxy" not found Jul 16 21:39:43.520612 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:43.520548 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:39:43.520612 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:43.520611 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:45.5205997 +0000 UTC m=+51.321486328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : secret "insights-runtime-extractor-tls" not found Jul 16 21:39:43.635069 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:43.635043 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pjkrj_4a85dd8e-53c8-408d-bd91-1a46d81a43fc/dns-node-resolver/0.log" Jul 16 21:39:44.635065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:44.635037 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd4s2_f870e849-9509-4e3a-9264-0ec11181dfb4/node-ca/0.log" Jul 16 21:39:45.239573 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.239529 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-slv9x_59fe218f-3f52-4ddb-b052-762f1e9ddef8/serve-healthcheck-canary/0.log" Jul 16 21:39:45.539187 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.539161 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:45.539359 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.539264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:45.539359 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.539309 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:39:45.539359 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.539325 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:39:45.539471 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.539372 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:49.53935749 +0000 UTC m=+55.340244119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : secret "insights-runtime-extractor-tls" not found Jul 16 21:39:45.539471 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.539388 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:49.539379749 +0000 UTC m=+55.340266378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : configmap "kube-rbac-proxy" not found Jul 16 21:39:45.942320 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:39:45.942320 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:39:45.942320 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942380 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942396 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942437 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942436 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls podName:f6571ef5-21df-43b3-b213-dd81d521d969 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942422123 +0000 UTC m=+67.743308755 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls") pod "dns-default-cv7nc" (UID: "f6571ef5-21df-43b3-b213-dd81d521d969") : secret "dns-default-metrics-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942478 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls podName:192d89ab-ec22-4974-a8b2-01f6eb9778fd nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942467299 +0000 UTC m=+67.743353928 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-xzmgw" (UID: "192d89ab-ec22-4974-a8b2-01f6eb9778fd") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942485 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942493 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942486817 +0000 UTC m=+67.743373446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : configmap references non-existent config key: service-ca.crt Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942502 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84bdd9894d-ftm6x: secret "image-registry-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942514 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert podName:293491f1-a543-4ae5-a8f9-ccdf2cd8957b nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942505134 +0000 UTC m=+67.743391764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-t25mv" (UID: "293491f1-a543-4ae5-a8f9-ccdf2cd8957b") : secret "networking-console-plugin-cert" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942579 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls podName:e020bb33-9484-4dcc-b073-2eb4e9e90666 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942533763 +0000 UTC m=+67.743420392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls") pod "image-registry-84bdd9894d-ftm6x" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666") : secret "image-registry-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:45.942666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942753 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:39:45.942908 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942756 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:39:45.943390 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942795 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls podName:88a3cdc3-9038-4524-8fce-d25d69418bc6 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942784889 +0000 UTC m=+67.743671520 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsx9n" (UID: "88a3cdc3-9038-4524-8fce-d25d69418bc6") : secret "samples-operator-tls" not found Jul 16 21:39:45.943390 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:45.942808 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs podName:ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0 nodeName:}" failed. No retries permitted until 2026-07-16 21:40:01.942801789 +0000 UTC m=+67.743688420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs") pod "router-default-6575ff56fb-h4b7b" (UID: "ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0") : secret "router-metrics-certs-default" not found Jul 16 21:39:49.572772 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:49.572737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:49.573328 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:49.572847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:49.573328 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:49.572889 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:39:49.573328 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:49.572984 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls podName:e93aebfb-7224-44b2-a908-43b5ef7d7897 nodeName:}" failed. No retries permitted until 2026-07-16 21:39:57.572970209 +0000 UTC m=+63.373856837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rzkg4" (UID: "e93aebfb-7224-44b2-a908-43b5ef7d7897") : secret "insights-runtime-extractor-tls" not found Jul 16 21:39:49.573439 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:49.573337 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e93aebfb-7224-44b2-a908-43b5ef7d7897-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:50.359887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:50.359853 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:50.360061 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:50.359908 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:39:50.360226 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:50.360214 2578 scope.go:117] "RemoveContainer" containerID="b20aa1d83425ddd4ead4f2afd6c71b6f0b4faff09236d8f184ef4ffa517661a0" Jul 16 21:39:50.360374 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:39:50.360356 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s4d5p_openshift-console-operator(b048d364-094e-443b-a259-7d836250a940)\"" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" podUID="b048d364-094e-443b-a259-7d836250a940" Jul 16 21:39:56.863963 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:56.863932 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gthgv" Jul 16 21:39:57.638917 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:57.638886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:57.641218 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:57.641195 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e93aebfb-7224-44b2-a908-43b5ef7d7897-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rzkg4\" (UID: \"e93aebfb-7224-44b2-a908-43b5ef7d7897\") " pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:57.686462 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:57.686434 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vpjhj\"" Jul 16 21:39:57.694525 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:57.694505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rzkg4" Jul 16 21:39:57.821315 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:57.821293 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rzkg4"] Jul 16 21:39:57.823916 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:39:57.823891 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93aebfb_7224_44b2_a908_43b5ef7d7897.slice/crio-936b5f575551ce09431872ef73f188e10ad8243800d9fa934a712fdf3bc48fb5 WatchSource:0}: Error finding container 936b5f575551ce09431872ef73f188e10ad8243800d9fa934a712fdf3bc48fb5: Status 404 returned error can't find the container with id 936b5f575551ce09431872ef73f188e10ad8243800d9fa934a712fdf3bc48fb5 Jul 16 21:39:57.992494 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:39:57.992421 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzkg4" event={"ID":"e93aebfb-7224-44b2-a908-43b5ef7d7897","Type":"ContainerStarted","Data":"936b5f575551ce09431872ef73f188e10ad8243800d9fa934a712fdf3bc48fb5"} Jul 16 21:40:00.466644 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.466612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:40:00.466644 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.466660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:40:00.469014 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.468990 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 21:40:00.469738 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.469717 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 21:40:00.479938 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.479910 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a9d655-6a74-4917-ab90-75eec7179b71-metrics-certs\") pod \"network-metrics-daemon-dclch\" (UID: \"63a9d655-6a74-4917-ab90-75eec7179b71\") " pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:40:00.480071 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.480048 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/853ab9b3-3a12-4858-a741-abb04892dd5b-original-pull-secret\") pod \"global-pull-secret-syncer-ddq22\" (UID: \"853ab9b3-3a12-4858-a741-abb04892dd5b\") " pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:40:00.490065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.490041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddq22" Jul 16 21:40:00.502371 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.502348 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bjvwm\"" Jul 16 21:40:00.511382 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.511363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dclch" Jul 16 21:40:00.567990 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.567958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:40:00.570870 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.570850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfkrj\" (UniqueName: \"kubernetes.io/projected/0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2-kube-api-access-jfkrj\") pod \"network-check-target-4cnbb\" (UID: \"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2\") " pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:40:00.810548 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.810514 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cd5nk\"" Jul 16 21:40:00.818594 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:00.818545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:40:01.669293 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.669262 2578 scope.go:117] "RemoveContainer" containerID="b20aa1d83425ddd4ead4f2afd6c71b6f0b4faff09236d8f184ef4ffa517661a0" Jul 16 21:40:01.981892 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.981846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:40:01.982065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.981947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:40:01.982065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.981992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:01.982065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.982023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:01.982065 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.982060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:40:01.982235 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.982096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:40:01.982235 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.982121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:01.983638 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.983606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-service-ca-bundle\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:01.986029 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.985892 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"image-registry-84bdd9894d-ftm6x\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:01.986239 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.986213 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88a3cdc3-9038-4524-8fce-d25d69418bc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsx9n\" (UID: \"88a3cdc3-9038-4524-8fce-d25d69418bc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:40:01.986423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.986389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6571ef5-21df-43b3-b213-dd81d521d969-metrics-tls\") pod \"dns-default-cv7nc\" (UID: \"f6571ef5-21df-43b3-b213-dd81d521d969\") " pod="openshift-dns/dns-default-cv7nc" Jul 16 21:40:01.986601 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.986580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0-metrics-certs\") pod \"router-default-6575ff56fb-h4b7b\" (UID: \"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0\") " pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:01.986696 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.986605 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/293491f1-a543-4ae5-a8f9-ccdf2cd8957b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-t25mv\" (UID: \"293491f1-a543-4ae5-a8f9-ccdf2cd8957b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:40:01.988394 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:01.987667 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/192d89ab-ec22-4974-a8b2-01f6eb9778fd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-xzmgw\" (UID: \"192d89ab-ec22-4974-a8b2-01f6eb9778fd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:40:02.166720 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.166685 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcgfg\"" Jul 16 21:40:02.174647 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.174619 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:02.184955 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.184934 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2qh2q\"" Jul 16 21:40:02.193806 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.193785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cv7nc" Jul 16 21:40:02.216090 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.216030 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gvkdk\"" Jul 16 21:40:02.223964 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.223939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" Jul 16 21:40:02.231344 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.231323 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4xjg2\"" Jul 16 21:40:02.239436 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.239420 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:02.246535 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.246517 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2c2f5\"" Jul 16 21:40:02.254627 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.254604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" Jul 16 21:40:02.260679 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.260658 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-p6tqr\"" Jul 16 21:40:02.269620 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.269602 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" Jul 16 21:40:02.833880 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.833764 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddq22"] Jul 16 21:40:02.970742 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:02.970628 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw"] Jul 16 21:40:03.009683 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.009650 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4cnbb"] Jul 16 21:40:03.031603 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.031465 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv"] Jul 16 21:40:03.031744 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.031689 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzkg4" event={"ID":"e93aebfb-7224-44b2-a908-43b5ef7d7897","Type":"ContainerStarted","Data":"629487ea03f63bb7dda4a7ced9c392c5f986070d5ed20d724d33290f63bc8d7b"} Jul 16 21:40:03.035894 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.035588 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" event={"ID":"192d89ab-ec22-4974-a8b2-01f6eb9778fd","Type":"ContainerStarted","Data":"ada61540fe7dd6d8aaf9dcea0b7a5fb10c867cb6a558fc84cc4677fedde35d9a"} Jul 16 21:40:03.038507 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.038432 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddq22" event={"ID":"853ab9b3-3a12-4858-a741-abb04892dd5b","Type":"ContainerStarted","Data":"a32e79843bcd20cb69b9172a0cc42e3582725eedb50951823b9c1ec7d023e57a"} Jul 16 21:40:03.050117 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.050092 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cv7nc"] Jul 16 21:40:03.054223 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.054163 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:40:03.054472 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.054392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" event={"ID":"b048d364-094e-443b-a259-7d836250a940","Type":"ContainerStarted","Data":"7295d27074effb18ad0f600833dc28f0d87b315abce9072e2e3bc9304808f329"} Jul 16 21:40:03.057149 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.057000 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:40:03.057411 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:03.057380 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6571ef5_21df_43b3_b213_dd81d521d969.slice/crio-99aaf39498ab62d3b3a90069b35b0e4766fe8aebc980a880e528bfb2b87e8183 WatchSource:0}: Error finding container 99aaf39498ab62d3b3a90069b35b0e4766fe8aebc980a880e528bfb2b87e8183: Status 404 returned error can't find the container with id 99aaf39498ab62d3b3a90069b35b0e4766fe8aebc980a880e528bfb2b87e8183 Jul 16 21:40:03.073204 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.073165 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" podStartSLOduration=43.281434281 podStartE2EDuration="53.073148857s" totalImagesPullingTime="9.791714576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:03.071658859 +0000 UTC m=+68.872545526" watchObservedRunningTime="2026-07-16 21:40:03.073148857 +0000 UTC m=+68.874035486" Jul 16 21:40:03.282420 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.282286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dclch"] Jul 16 21:40:03.285222 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:03.285195 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a9d655_6a74_4917_ab90_75eec7179b71.slice/crio-eaab852d1f7b5070ceea40ef0487b50272193dd88b12c60aa5c3c285e3a7b0b5 WatchSource:0}: Error finding container eaab852d1f7b5070ceea40ef0487b50272193dd88b12c60aa5c3c285e3a7b0b5: Status 404 returned error can't find the container with id eaab852d1f7b5070ceea40ef0487b50272193dd88b12c60aa5c3c285e3a7b0b5 Jul 16 21:40:03.288755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.286192 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6575ff56fb-h4b7b"] Jul 16 21:40:03.288755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.287451 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84bdd9894d-ftm6x"] Jul 16 21:40:03.288755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.288585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n"] Jul 16 21:40:03.290279 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:03.290255 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode020bb33_9484_4dcc_b073_2eb4e9e90666.slice/crio-e4eaf0aadf68fb6d726a544560e555e603d664c72b6692d64a76d4a2f61ab8bd WatchSource:0}: Error finding container e4eaf0aadf68fb6d726a544560e555e603d664c72b6692d64a76d4a2f61ab8bd: Status 404 returned error can't find the container with id e4eaf0aadf68fb6d726a544560e555e603d664c72b6692d64a76d4a2f61ab8bd Jul 16 21:40:03.290709 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:03.290680 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3df8b6_f0a0_4fc4_82be_5bd5dc5ee3e0.slice/crio-dd101dcbdb8b383979ab4a3022aef67f736d8b610110e793e26e18b61c86ab41 WatchSource:0}: Error finding container dd101dcbdb8b383979ab4a3022aef67f736d8b610110e793e26e18b61c86ab41: Status 404 returned error can't find the container with id dd101dcbdb8b383979ab4a3022aef67f736d8b610110e793e26e18b61c86ab41 Jul 16 21:40:03.687044 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:03.686957 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-s4d5p" Jul 16 21:40:04.061547 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.061457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dclch" event={"ID":"63a9d655-6a74-4917-ab90-75eec7179b71","Type":"ContainerStarted","Data":"eaab852d1f7b5070ceea40ef0487b50272193dd88b12c60aa5c3c285e3a7b0b5"} Jul 16 21:40:04.065159 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.065088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cv7nc" event={"ID":"f6571ef5-21df-43b3-b213-dd81d521d969","Type":"ContainerStarted","Data":"99aaf39498ab62d3b3a90069b35b0e4766fe8aebc980a880e528bfb2b87e8183"} Jul 16 21:40:04.070725 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.070645 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" event={"ID":"88a3cdc3-9038-4524-8fce-d25d69418bc6","Type":"ContainerStarted","Data":"acdd412c7329377306a65927984deadd82508d7f03753c1a78ffde753de18c83"} Jul 16 21:40:04.075945 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.075074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" event={"ID":"e020bb33-9484-4dcc-b073-2eb4e9e90666","Type":"ContainerStarted","Data":"4d98254444264bf5347aeba19fd22fdf490870d1ef2bb445b96057d81a2f30b4"} Jul 16 21:40:04.075945 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.075118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" event={"ID":"e020bb33-9484-4dcc-b073-2eb4e9e90666","Type":"ContainerStarted","Data":"e4eaf0aadf68fb6d726a544560e555e603d664c72b6692d64a76d4a2f61ab8bd"} Jul 16 21:40:04.075945 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.075897 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:04.077833 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.077776 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" event={"ID":"293491f1-a543-4ae5-a8f9-ccdf2cd8957b","Type":"ContainerStarted","Data":"da22ab2a2e411b4f4236cfaf5f10ca77cef5c93af34de632a53b677b87ffb13e"} Jul 16 21:40:04.083228 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.083168 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4cnbb" event={"ID":"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2","Type":"ContainerStarted","Data":"cec201cb9f8030e78e2a1f74eaa59c85fe0a4c2222cb0c8f59d174fcbb32f1ee"} Jul 16 21:40:04.083228 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.083212 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4cnbb" event={"ID":"0acbfad5-3bda-4e83-a9f7-b5a3c2eb76c2","Type":"ContainerStarted","Data":"4a8da70e9ff6def7b7b28ed4b138b1f60952ad10a6a30c39fc1d64f6b8710f0b"} Jul 16 21:40:04.083509 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.083487 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:40:04.087072 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.086995 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" event={"ID":"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0","Type":"ContainerStarted","Data":"822f41f09b4993dc25cfef44f8c95e7cafb21b16d4a704de5549176409a3a858"} Jul 16 21:40:04.087072 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.087033 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" event={"ID":"ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0","Type":"ContainerStarted","Data":"dd101dcbdb8b383979ab4a3022aef67f736d8b610110e793e26e18b61c86ab41"} Jul 16 21:40:04.096750 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.095533 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" podStartSLOduration=69.095520428 podStartE2EDuration="1m9.095520428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:04.095084751 +0000 UTC m=+69.895971404" watchObservedRunningTime="2026-07-16 21:40:04.095520428 +0000 UTC m=+69.896407059" Jul 16 21:40:04.112599 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.112527 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4cnbb" podStartSLOduration=69.112505798 podStartE2EDuration="1m9.112505798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:04.111223187 +0000 UTC m=+69.912109839" watchObservedRunningTime="2026-07-16 21:40:04.112505798 +0000 UTC m=+69.913392450" Jul 16 21:40:04.134005 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.132712 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" podStartSLOduration=54.132699622 podStartE2EDuration="54.132699622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:04.132229602 +0000 UTC m=+69.933116273" watchObservedRunningTime="2026-07-16 21:40:04.132699622 +0000 UTC m=+69.933586273" Jul 16 21:40:04.240049 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.239859 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:04.243298 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:04.243064 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:05.093660 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:05.092996 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:05.094423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:05.094302 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6575ff56fb-h4b7b" Jul 16 21:40:07.964464 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:07.964432 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7"] Jul 16 21:40:07.987451 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:07.987417 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:07.994782 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:07.993425 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7"] Jul 16 21:40:07.994782 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:07.994469 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 21:40:07.994966 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:07.994850 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 21:40:07.996176 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:07.995950 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 21:40:08.000239 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.000215 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 21:40:08.048749 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.048721 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-crmdz"] Jul 16 21:40:08.070837 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.070810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:08.073968 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.073944 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-crmdz"] Jul 16 21:40:08.074764 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.074483 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 21:40:08.074764 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.074535 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 21:40:08.080746 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.080719 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5rd8p\"" Jul 16 21:40:08.137883 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.137848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc91e17c-ab22-48fc-8078-1dba77afef68-tmp\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.138060 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.137897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-784bd\" (UniqueName: \"kubernetes.io/projected/bfa93aad-a92d-4020-82a2-53541ba23065-kube-api-access-784bd\") pod \"downloads-5bc67dcd46-crmdz\" (UID: \"bfa93aad-a92d-4020-82a2-53541ba23065\") " pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:08.138060 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.137971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84x4c\" (UniqueName: \"kubernetes.io/projected/dc91e17c-ab22-48fc-8078-1dba77afef68-kube-api-access-84x4c\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.138060 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.138009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/dc91e17c-ab22-48fc-8078-1dba77afef68-klusterlet-config\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.238976 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.238893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc91e17c-ab22-48fc-8078-1dba77afef68-tmp\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.238976 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.238939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-784bd\" (UniqueName: \"kubernetes.io/projected/bfa93aad-a92d-4020-82a2-53541ba23065-kube-api-access-784bd\") pod \"downloads-5bc67dcd46-crmdz\" (UID: \"bfa93aad-a92d-4020-82a2-53541ba23065\") " pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:08.239192 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.239005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84x4c\" (UniqueName: \"kubernetes.io/projected/dc91e17c-ab22-48fc-8078-1dba77afef68-kube-api-access-84x4c\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.239192 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.239047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/dc91e17c-ab22-48fc-8078-1dba77afef68-klusterlet-config\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.239386 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.239357 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc91e17c-ab22-48fc-8078-1dba77afef68-tmp\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.241987 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.241961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/dc91e17c-ab22-48fc-8078-1dba77afef68-klusterlet-config\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.249306 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.249283 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-784bd\" (UniqueName: \"kubernetes.io/projected/bfa93aad-a92d-4020-82a2-53541ba23065-kube-api-access-784bd\") pod \"downloads-5bc67dcd46-crmdz\" (UID: \"bfa93aad-a92d-4020-82a2-53541ba23065\") " pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:08.252244 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.252193 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84x4c\" (UniqueName: \"kubernetes.io/projected/dc91e17c-ab22-48fc-8078-1dba77afef68-kube-api-access-84x4c\") pod \"klusterlet-addon-workmgr-cc7df879c-v2wr7\" (UID: \"dc91e17c-ab22-48fc-8078-1dba77afef68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.309037 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.309008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:08.383872 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:08.383833 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:12.121371 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:12.121343 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-crmdz"] Jul 16 21:40:12.193128 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:12.193102 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa93aad_a92d_4020_82a2_53541ba23065.slice/crio-81c7b810c630761ad5fc3d688282c77f207aa99dec7d81ce3aca1ef727c2fac4 WatchSource:0}: Error finding container 81c7b810c630761ad5fc3d688282c77f207aa99dec7d81ce3aca1ef727c2fac4: Status 404 returned error can't find the container with id 81c7b810c630761ad5fc3d688282c77f207aa99dec7d81ce3aca1ef727c2fac4 Jul 16 21:40:12.378343 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:12.378317 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7"] Jul 16 21:40:13.124176 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.124137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzkg4" event={"ID":"e93aebfb-7224-44b2-a908-43b5ef7d7897","Type":"ContainerStarted","Data":"b3c169c90751000e9f1bc8eae78a065f22d0be17427ee0f155119809518eab3e"} Jul 16 21:40:13.126739 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.126246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cv7nc" event={"ID":"f6571ef5-21df-43b3-b213-dd81d521d969","Type":"ContainerStarted","Data":"29b4eb2e550f36af78a83f16a28793f09baec2e42b44b8b0312852a746a4e1d0"} Jul 16 21:40:13.126739 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.126294 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cv7nc" event={"ID":"f6571ef5-21df-43b3-b213-dd81d521d969","Type":"ContainerStarted","Data":"4604107af91220f7575d7d3d879abfb23d2c1fe64b599e54a9ababbaa6387cf9"} Jul 16 21:40:13.126739 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.126361 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cv7nc" Jul 16 21:40:13.128530 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.128503 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" event={"ID":"293491f1-a543-4ae5-a8f9-ccdf2cd8957b","Type":"ContainerStarted","Data":"e3ebc2f62f92722f9eb2d0ed5492831420fe2760ed07f63985c389f14b1e40d8"} Jul 16 21:40:13.129947 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.129920 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-crmdz" event={"ID":"bfa93aad-a92d-4020-82a2-53541ba23065","Type":"ContainerStarted","Data":"81c7b810c630761ad5fc3d688282c77f207aa99dec7d81ce3aca1ef727c2fac4"} Jul 16 21:40:13.132148 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.132126 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" event={"ID":"88a3cdc3-9038-4524-8fce-d25d69418bc6","Type":"ContainerStarted","Data":"b30f4a5d9b1b658d25564847aab12d8cdbe84e5126c0ef55d4a2619b90a148a2"} Jul 16 21:40:13.132251 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.132184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" event={"ID":"88a3cdc3-9038-4524-8fce-d25d69418bc6","Type":"ContainerStarted","Data":"d8b90cd7e97372ef55791febdb912f40683c2000b423dae9e328636b361ab9ca"} Jul 16 21:40:13.134085 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.134060 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" event={"ID":"192d89ab-ec22-4974-a8b2-01f6eb9778fd","Type":"ContainerStarted","Data":"1f4aa7b9b242aed865410136009a63dc7f914c4a75808d6cfb07b579d45c9f74"} Jul 16 21:40:13.136396 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.136371 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddq22" event={"ID":"853ab9b3-3a12-4858-a741-abb04892dd5b","Type":"ContainerStarted","Data":"9d03bff87acaa80719a6a45ca832294960800a5e5301aced8404e1761f6c036f"} Jul 16 21:40:13.138475 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.138450 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dclch" event={"ID":"63a9d655-6a74-4917-ab90-75eec7179b71","Type":"ContainerStarted","Data":"29fd332533cd82a9f803809a37ec3a5950bb4fa635da986028ba6ac3a28a5f35"} Jul 16 21:40:13.138595 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.138507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dclch" event={"ID":"63a9d655-6a74-4917-ab90-75eec7179b71","Type":"ContainerStarted","Data":"c2ae3aa61eba99eac3ddd9aad6c2dd83caccc311f098d803ff278d08b1f63dea"} Jul 16 21:40:13.139729 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.139708 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" event={"ID":"dc91e17c-ab22-48fc-8078-1dba77afef68","Type":"ContainerStarted","Data":"b3b02846b97c335482842812cb357da74b3f1a8f80f475b85e98c0c96c1565d6"} Jul 16 21:40:13.144411 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.144376 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cv7nc" podStartSLOduration=34.013773965 podStartE2EDuration="43.144362571s" totalImagesPullingTime="9.130588606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:13.14268685 +0000 UTC m=+78.943573522" watchObservedRunningTime="2026-07-16 21:40:13.144362571 +0000 UTC m=+78.945249222" Jul 16 21:40:13.163676 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.163631 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsx9n" podStartSLOduration=54.248737706 podStartE2EDuration="1m3.163596376s" totalImagesPullingTime="8.91485867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:13.162336476 +0000 UTC m=+78.963223128" watchObservedRunningTime="2026-07-16 21:40:13.163596376 +0000 UTC m=+78.964483030" Jul 16 21:40:13.177514 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.176985 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dclch" podStartSLOduration=70.364184521 podStartE2EDuration="1m18.176968927s" totalImagesPullingTime="7.812784406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:13.176744965 +0000 UTC m=+78.977631617" watchObservedRunningTime="2026-07-16 21:40:13.176968927 +0000 UTC m=+78.977855579" Jul 16 21:40:13.195759 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.195713 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-xzmgw" podStartSLOduration=53.987811584 podStartE2EDuration="1m3.195669698s" totalImagesPullingTime="9.207858114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:13.195034107 +0000 UTC m=+78.995920776" watchObservedRunningTime="2026-07-16 21:40:13.195669698 +0000 UTC m=+78.996556351" Jul 16 21:40:13.212939 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.212881 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-t25mv" podStartSLOduration=41.157438073 podStartE2EDuration="49.21286122s" totalImagesPullingTime="8.055423147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:13.211838434 +0000 UTC m=+79.012725085" watchObservedRunningTime="2026-07-16 21:40:13.21286122 +0000 UTC m=+79.013747871" Jul 16 21:40:13.234797 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:13.234151 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ddq22" podStartSLOduration=69.087799114 podStartE2EDuration="1m18.234131079s" totalImagesPullingTime="9.146331965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:13.232350005 +0000 UTC m=+79.033236681" watchObservedRunningTime="2026-07-16 21:40:13.234131079 +0000 UTC m=+79.035017732" Jul 16 21:40:19.172781 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:19.172691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" event={"ID":"dc91e17c-ab22-48fc-8078-1dba77afef68","Type":"ContainerStarted","Data":"b6d0891f5518185033328b174925b3c3498779ab66d4a643cff0bcb357155f72"} Jul 16 21:40:19.173189 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:19.172916 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:19.175515 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:19.175476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" Jul 16 21:40:19.176663 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:19.176630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rzkg4" event={"ID":"e93aebfb-7224-44b2-a908-43b5ef7d7897","Type":"ContainerStarted","Data":"81cf5a993620ce6963162979abe486395ba6ec12a089f7300605875d33fc6226"} Jul 16 21:40:19.191507 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:19.191459 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc7df879c-v2wr7" podStartSLOduration=5.663888554 podStartE2EDuration="12.19144442s" totalImagesPullingTime="6.527555866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:19.190835729 +0000 UTC m=+84.991722383" watchObservedRunningTime="2026-07-16 21:40:19.19144442 +0000 UTC m=+84.992331072" Jul 16 21:40:19.211634 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:19.211589 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rzkg4" podStartSLOduration=17.563710753 podStartE2EDuration="38.211570061s" totalImagesPullingTime="20.647859308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:39:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:19.211270196 +0000 UTC m=+85.012156867" watchObservedRunningTime="2026-07-16 21:40:19.211570061 +0000 UTC m=+85.012456708" Jul 16 21:40:22.180814 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:22.180778 2578 patch_prober.go:28] interesting pod/image-registry-84bdd9894d-ftm6x container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 21:40:22.181351 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:22.180849 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" podUID="e020bb33-9484-4dcc-b073-2eb4e9e90666" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:40:23.152087 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:23.152053 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cv7nc" Jul 16 21:40:25.337902 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.337867 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nhmnb"] Jul 16 21:40:25.342810 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.342783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.345330 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.345308 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 21:40:25.345916 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.345895 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 21:40:25.346352 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.346328 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pm9ql\"" Jul 16 21:40:25.346468 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.346399 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 21:40:25.346468 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.346405 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 21:40:25.485661 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7228962-c722-49df-93a4-de1a9f880d97-metrics-client-ca\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.485661 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqxdr\" (UniqueName: \"kubernetes.io/projected/f7228962-c722-49df-93a4-de1a9f880d97-kube-api-access-qqxdr\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.485908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-root\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.485908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.485908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-sys\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.485908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-textfile\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.486095 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-wtmp\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.486095 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.485974 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-tls\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.486095 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.486028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-accelerators-collector-config\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.586920 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.586890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7228962-c722-49df-93a4-de1a9f880d97-metrics-client-ca\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.586920 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.586925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqxdr\" (UniqueName: \"kubernetes.io/projected/f7228962-c722-49df-93a4-de1a9f880d97-kube-api-access-qqxdr\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587147 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587101 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-root\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587147 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587242 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587163 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-sys\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587242 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-textfile\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587242 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587205 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-root\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587242 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-wtmp\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587432 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-tls\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587432 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587313 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-sys\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587432 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-accelerators-collector-config\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587432 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587349 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-wtmp\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587685 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587638 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7228962-c722-49df-93a4-de1a9f880d97-metrics-client-ca\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587817 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587706 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-textfile\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.587875 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.587843 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-accelerators-collector-config\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.590144 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.590092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-tls\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.590430 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.590399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7228962-c722-49df-93a4-de1a9f880d97-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.595236 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.595214 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqxdr\" (UniqueName: \"kubernetes.io/projected/f7228962-c722-49df-93a4-de1a9f880d97-kube-api-access-qqxdr\") pod \"node-exporter-nhmnb\" (UID: \"f7228962-c722-49df-93a4-de1a9f880d97\") " pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:25.654163 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:25.654128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nhmnb" Jul 16 21:40:26.101133 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:26.101098 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:31.292182 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:31.292136 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7228962_c722_49df_93a4_de1a9f880d97.slice/crio-81ef2a8fa9f2ca4455bcfd91cf912b0d71815b27f6c2aab7a9481647e90c7afe WatchSource:0}: Error finding container 81ef2a8fa9f2ca4455bcfd91cf912b0d71815b27f6c2aab7a9481647e90c7afe: Status 404 returned error can't find the container with id 81ef2a8fa9f2ca4455bcfd91cf912b0d71815b27f6c2aab7a9481647e90c7afe Jul 16 21:40:32.119343 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:32.119313 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84bdd9894d-ftm6x"] Jul 16 21:40:32.236085 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:32.236040 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nhmnb" event={"ID":"f7228962-c722-49df-93a4-de1a9f880d97","Type":"ContainerStarted","Data":"81ef2a8fa9f2ca4455bcfd91cf912b0d71815b27f6c2aab7a9481647e90c7afe"} Jul 16 21:40:32.237759 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:32.237730 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-crmdz" event={"ID":"bfa93aad-a92d-4020-82a2-53541ba23065","Type":"ContainerStarted","Data":"3a4df6d6d9f00d6e9dc5a98618c02e471596dad03185ac8ff0ddaa2d19e1501e"} Jul 16 21:40:32.238584 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:32.238544 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:32.250706 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:32.250683 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-crmdz" Jul 16 21:40:32.295642 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:32.293073 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-crmdz" podStartSLOduration=4.84564322 podStartE2EDuration="24.293048298s" totalImagesPullingTime="19.447405078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:32.259195317 +0000 UTC m=+98.060081968" watchObservedRunningTime="2026-07-16 21:40:32.293048298 +0000 UTC m=+98.093934955" Jul 16 21:40:33.244474 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:33.244389 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7228962-c722-49df-93a4-de1a9f880d97" containerID="6ead7f63b6e1d9fc84bbdaae6269cca8821047d309e905f6438215c769fc8ce6" exitCode=0 Jul 16 21:40:33.244670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:33.244485 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nhmnb" event={"ID":"f7228962-c722-49df-93a4-de1a9f880d97","Type":"ContainerDied","Data":"6ead7f63b6e1d9fc84bbdaae6269cca8821047d309e905f6438215c769fc8ce6"} Jul 16 21:40:34.251894 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:34.251858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nhmnb" event={"ID":"f7228962-c722-49df-93a4-de1a9f880d97","Type":"ContainerStarted","Data":"49d5091b85f15df9c825ba0632ccc0f4f50a2059b8d918d1e8285d9df2439460"} Jul 16 21:40:34.252352 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:34.251924 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nhmnb" event={"ID":"f7228962-c722-49df-93a4-de1a9f880d97","Type":"ContainerStarted","Data":"b2d1bcde624c8c9ae8ceefabe5a2547b64144b85dc9186353594681600dc3449"} Jul 16 21:40:34.273131 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:34.273076 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nhmnb" podStartSLOduration=7.5965396389999995 podStartE2EDuration="9.27305571s" totalImagesPullingTime="1.589247268s" totalInitContainerRuntime="87.268803ms" isStatefulPod=true podCreationTimestamp="2026-07-16 21:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:34.271979797 +0000 UTC m=+100.072866448" watchObservedRunningTime="2026-07-16 21:40:34.27305571 +0000 UTC m=+100.073942361" Jul 16 21:40:35.095721 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.095690 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4cnbb" Jul 16 21:40:35.901538 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.901503 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b9c6879d9-klztl"] Jul 16 21:40:35.941299 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.941259 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b9c6879d9-klztl"] Jul 16 21:40:35.941455 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.941368 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.944107 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.944078 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 21:40:35.944240 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.944078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rq4nh\"" Jul 16 21:40:35.944633 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.944613 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 21:40:35.945100 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.945080 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 21:40:35.945273 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.945256 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 21:40:35.945924 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.945906 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 21:40:35.949624 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.949604 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 21:40:35.975483 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975454 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-console-config\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.975608 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-oauth-serving-cert\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.975608 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-service-ca\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.975608 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-trusted-ca-bundle\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.975783 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-oauth-config\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.975783 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975651 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kbb\" (UniqueName: \"kubernetes.io/projected/2afa8750-55df-4d63-865d-74a16e1cbb11-kube-api-access-96kbb\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:35.975783 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:35.975722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-serving-cert\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076332 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-oauth-config\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076470 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kbb\" (UniqueName: \"kubernetes.io/projected/2afa8750-55df-4d63-865d-74a16e1cbb11-kube-api-access-96kbb\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076539 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-serving-cert\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076626 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-console-config\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076626 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-oauth-serving-cert\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076734 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-service-ca\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.076734 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.076673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-trusted-ca-bundle\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.077418 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.077391 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-console-config\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.077537 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.077478 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-oauth-serving-cert\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.077912 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.077887 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-trusted-ca-bundle\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.078085 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.078068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-service-ca\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.079383 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.079363 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-oauth-config\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.080060 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.080037 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-serving-cert\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.085578 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.085531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kbb\" (UniqueName: \"kubernetes.io/projected/2afa8750-55df-4d63-865d-74a16e1cbb11-kube-api-access-96kbb\") pod \"console-5b9c6879d9-klztl\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.253156 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.253079 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:36.440779 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:36.440747 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b9c6879d9-klztl"] Jul 16 21:40:36.448664 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:40:36.448634 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2afa8750_55df_4d63_865d_74a16e1cbb11.slice/crio-21b525ea43366a45c38db21977744e79e11acdcae70a5a2b65099a65f6a098f4 WatchSource:0}: Error finding container 21b525ea43366a45c38db21977744e79e11acdcae70a5a2b65099a65f6a098f4: Status 404 returned error can't find the container with id 21b525ea43366a45c38db21977744e79e11acdcae70a5a2b65099a65f6a098f4 Jul 16 21:40:37.267838 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:37.267791 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c6879d9-klztl" event={"ID":"2afa8750-55df-4d63-865d-74a16e1cbb11","Type":"ContainerStarted","Data":"21b525ea43366a45c38db21977744e79e11acdcae70a5a2b65099a65f6a098f4"} Jul 16 21:40:41.291369 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:41.291329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c6879d9-klztl" event={"ID":"2afa8750-55df-4d63-865d-74a16e1cbb11","Type":"ContainerStarted","Data":"ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4"} Jul 16 21:40:41.310140 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:41.310083 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b9c6879d9-klztl" podStartSLOduration=2.404336151 podStartE2EDuration="6.310062877s" totalImagesPullingTime="3.905726726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:40:41.308749203 +0000 UTC m=+107.109635855" watchObservedRunningTime="2026-07-16 21:40:41.310062877 +0000 UTC m=+107.110949529" Jul 16 21:40:46.253713 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:46.253681 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:46.254136 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:46.253752 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:46.258574 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:46.258528 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:46.312344 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:46.312317 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:40:57.150518 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.150438 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" podUID="e020bb33-9484-4dcc-b073-2eb4e9e90666" containerName="registry" containerID="cri-o://4d98254444264bf5347aeba19fd22fdf490870d1ef2bb445b96057d81a2f30b4" gracePeriod=30 Jul 16 21:40:57.349397 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.349368 2578 generic.go:356] "Generic (PLEG): container finished" podID="e020bb33-9484-4dcc-b073-2eb4e9e90666" containerID="4d98254444264bf5347aeba19fd22fdf490870d1ef2bb445b96057d81a2f30b4" exitCode=0 Jul 16 21:40:57.349525 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.349446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" event={"ID":"e020bb33-9484-4dcc-b073-2eb4e9e90666","Type":"ContainerDied","Data":"4d98254444264bf5347aeba19fd22fdf490870d1ef2bb445b96057d81a2f30b4"} Jul 16 21:40:57.419353 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.419233 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:57.554785 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554756 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e020bb33-9484-4dcc-b073-2eb4e9e90666-ca-trust-extracted\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554802 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-certificates\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554831 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-image-registry-private-configuration\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554885 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554912 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-trusted-ca\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554936 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-bound-sa-token\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554969 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-installation-pull-secrets\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.554993 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.554991 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7d44\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-kube-api-access-d7d44\") pod \"e020bb33-9484-4dcc-b073-2eb4e9e90666\" (UID: \"e020bb33-9484-4dcc-b073-2eb4e9e90666\") " Jul 16 21:40:57.555326 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.555282 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:40:57.555386 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.555344 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:40:57.557505 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.557469 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:40:57.557703 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.557683 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-kube-api-access-d7d44" (OuterVolumeSpecName: "kube-api-access-d7d44") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "kube-api-access-d7d44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:40:57.557759 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.557729 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:40:57.557800 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.557778 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:40:57.557901 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.557885 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:40:57.563306 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.563283 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e020bb33-9484-4dcc-b073-2eb4e9e90666-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e020bb33-9484-4dcc-b073-2eb4e9e90666" (UID: "e020bb33-9484-4dcc-b073-2eb4e9e90666"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:40:57.656504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656471 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656497 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-trusted-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656509 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-bound-sa-token\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656521 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-installation-pull-secrets\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656534 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7d44\" (UniqueName: \"kubernetes.io/projected/e020bb33-9484-4dcc-b073-2eb4e9e90666-kube-api-access-d7d44\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656546 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e020bb33-9484-4dcc-b073-2eb4e9e90666-ca-trust-extracted\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656582 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e020bb33-9484-4dcc-b073-2eb4e9e90666-registry-certificates\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:57.656755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:57.656595 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e020bb33-9484-4dcc-b073-2eb4e9e90666-image-registry-private-configuration\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:40:58.354296 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:58.354259 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" Jul 16 21:40:58.354296 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:58.354276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84bdd9894d-ftm6x" event={"ID":"e020bb33-9484-4dcc-b073-2eb4e9e90666","Type":"ContainerDied","Data":"e4eaf0aadf68fb6d726a544560e555e603d664c72b6692d64a76d4a2f61ab8bd"} Jul 16 21:40:58.354875 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:58.354341 2578 scope.go:117] "RemoveContainer" containerID="4d98254444264bf5347aeba19fd22fdf490870d1ef2bb445b96057d81a2f30b4" Jul 16 21:40:58.376646 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:58.376624 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84bdd9894d-ftm6x"] Jul 16 21:40:58.380802 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:58.380783 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84bdd9894d-ftm6x"] Jul 16 21:40:58.675069 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:40:58.674994 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e020bb33-9484-4dcc-b073-2eb4e9e90666" path="/var/lib/kubelet/pods/e020bb33-9484-4dcc-b073-2eb4e9e90666/volumes" Jul 16 21:41:02.371271 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:02.371235 2578 generic.go:356] "Generic (PLEG): container finished" podID="c6594bdb-a475-4973-8068-f6bf5ddd341a" containerID="fa477787b1bcddb55fc88381e10cfa5906efa8889b488a791b4a1d93c613c8be" exitCode=0 Jul 16 21:41:02.371715 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:02.371281 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" event={"ID":"c6594bdb-a475-4973-8068-f6bf5ddd341a","Type":"ContainerDied","Data":"fa477787b1bcddb55fc88381e10cfa5906efa8889b488a791b4a1d93c613c8be"} Jul 16 21:41:02.371715 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:02.371545 2578 scope.go:117] "RemoveContainer" containerID="fa477787b1bcddb55fc88381e10cfa5906efa8889b488a791b4a1d93c613c8be" Jul 16 21:41:03.376699 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:03.376666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-5mx67" event={"ID":"c6594bdb-a475-4973-8068-f6bf5ddd341a","Type":"ContainerStarted","Data":"7c4cc4466bf96565681ff3ceb8aaf0c3b931a8b39989e2c68b00c559147acf4b"} Jul 16 21:41:11.406200 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:11.406170 2578 generic.go:356] "Generic (PLEG): container finished" podID="b078fc92-aee9-4bf5-a3b6-bce92c892808" containerID="a29540f5c211d1664a4b5544d631e59f6dd1f3ab17c36edfd139105760967cb9" exitCode=0 Jul 16 21:41:11.406580 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:11.406246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-62gc6" event={"ID":"b078fc92-aee9-4bf5-a3b6-bce92c892808","Type":"ContainerDied","Data":"a29540f5c211d1664a4b5544d631e59f6dd1f3ab17c36edfd139105760967cb9"} Jul 16 21:41:11.406580 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:11.406523 2578 scope.go:117] "RemoveContainer" containerID="a29540f5c211d1664a4b5544d631e59f6dd1f3ab17c36edfd139105760967cb9" Jul 16 21:41:12.411748 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:12.411711 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-62gc6" event={"ID":"b078fc92-aee9-4bf5-a3b6-bce92c892808","Type":"ContainerStarted","Data":"915e0031ac8c35ba76e28850f498397c4bdff08ac6844b6f5049b6cdf70eaee2"} Jul 16 21:41:12.413301 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:12.413274 2578 generic.go:356] "Generic (PLEG): container finished" podID="3fe2c709-066d-4f62-a8f6-b191620dcf80" containerID="95fc303398cc79518c91e4fe01cd862d346a062a63eea88ea0f2e1cf28e72d53" exitCode=0 Jul 16 21:41:12.413428 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:12.413315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" event={"ID":"3fe2c709-066d-4f62-a8f6-b191620dcf80","Type":"ContainerDied","Data":"95fc303398cc79518c91e4fe01cd862d346a062a63eea88ea0f2e1cf28e72d53"} Jul 16 21:41:12.413596 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:12.413582 2578 scope.go:117] "RemoveContainer" containerID="95fc303398cc79518c91e4fe01cd862d346a062a63eea88ea0f2e1cf28e72d53" Jul 16 21:41:13.419712 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:41:13.419677 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2lsl4" event={"ID":"3fe2c709-066d-4f62-a8f6-b191620dcf80","Type":"ContainerStarted","Data":"7c0b23f98138ccd482046baabefcf16fc8316ea9eca55d12e6a5f701d75c56e5"} Jul 16 21:42:04.075468 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.075439 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cd4899969-c8hhp"] Jul 16 21:42:04.075901 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.075839 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e020bb33-9484-4dcc-b073-2eb4e9e90666" containerName="registry" Jul 16 21:42:04.075901 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.075854 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e020bb33-9484-4dcc-b073-2eb4e9e90666" containerName="registry" Jul 16 21:42:04.075901 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.075903 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e020bb33-9484-4dcc-b073-2eb4e9e90666" containerName="registry" Jul 16 21:42:04.082333 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.082308 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.091287 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.091260 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd4899969-c8hhp"] Jul 16 21:42:04.105037 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-trusted-ca-bundle\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.105166 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105060 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-config\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.105166 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-service-ca\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.105166 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-oauth-serving-cert\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.105325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-oauth-config\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.105325 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-serving-cert\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.105426 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.105337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccbk6\" (UniqueName: \"kubernetes.io/projected/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-kube-api-access-ccbk6\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206247 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-serving-cert\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206389 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccbk6\" (UniqueName: \"kubernetes.io/projected/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-kube-api-access-ccbk6\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206389 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-trusted-ca-bundle\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206389 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-config\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206389 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-service-ca\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206389 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-oauth-serving-cert\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.206704 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.206428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-oauth-config\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.207189 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.207158 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-oauth-serving-cert\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.207307 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.207158 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-config\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.207307 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.207288 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-trusted-ca-bundle\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.207509 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.207491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-service-ca\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.208819 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.208801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-oauth-config\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.208934 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.208916 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-serving-cert\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.216049 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.216025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccbk6\" (UniqueName: \"kubernetes.io/projected/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-kube-api-access-ccbk6\") pod \"console-6cd4899969-c8hhp\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.392771 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.392708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:04.525348 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.525320 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd4899969-c8hhp"] Jul 16 21:42:04.528359 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:42:04.528336 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da38d7e_21f1_4cfd_b490_f95a4500d8a7.slice/crio-05bf10e420ec8a07f39a31c34dadc70a7b1e43ffbe465346bd704db5e0c58e32 WatchSource:0}: Error finding container 05bf10e420ec8a07f39a31c34dadc70a7b1e43ffbe465346bd704db5e0c58e32: Status 404 returned error can't find the container with id 05bf10e420ec8a07f39a31c34dadc70a7b1e43ffbe465346bd704db5e0c58e32 Jul 16 21:42:04.618899 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.618858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd4899969-c8hhp" event={"ID":"1da38d7e-21f1-4cfd-b490-f95a4500d8a7","Type":"ContainerStarted","Data":"652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91"} Jul 16 21:42:04.619035 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.618914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd4899969-c8hhp" event={"ID":"1da38d7e-21f1-4cfd-b490-f95a4500d8a7","Type":"ContainerStarted","Data":"05bf10e420ec8a07f39a31c34dadc70a7b1e43ffbe465346bd704db5e0c58e32"} Jul 16 21:42:04.637641 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:04.637592 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cd4899969-c8hhp" podStartSLOduration=0.637577114 podStartE2EDuration="637.577114ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:42:04.63594644 +0000 UTC m=+190.436833118" watchObservedRunningTime="2026-07-16 21:42:04.637577114 +0000 UTC m=+190.438463756" Jul 16 21:42:14.393210 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:14.393171 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:14.393712 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:14.393265 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:14.398003 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:14.397984 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:14.658088 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:14.658016 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:42:14.705739 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:14.705704 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b9c6879d9-klztl"] Jul 16 21:42:39.734656 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:39.734602 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b9c6879d9-klztl" podUID="2afa8750-55df-4d63-865d-74a16e1cbb11" containerName="console" containerID="cri-o://ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4" gracePeriod=15 Jul 16 21:42:39.978670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:39.978649 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:42:40.061034 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061007 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-service-ca\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061198 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061042 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-oauth-config\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061198 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061096 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96kbb\" (UniqueName: \"kubernetes.io/projected/2afa8750-55df-4d63-865d-74a16e1cbb11-kube-api-access-96kbb\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061316 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061246 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-oauth-serving-cert\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061316 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061272 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-serving-cert\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061316 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061304 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-console-config\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061467 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061329 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-trusted-ca-bundle\") pod \"2afa8750-55df-4d63-865d-74a16e1cbb11\" (UID: \"2afa8750-55df-4d63-865d-74a16e1cbb11\") " Jul 16 21:42:40.061611 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061544 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-service-ca" (OuterVolumeSpecName: "service-ca") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:42:40.061709 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061622 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:42:40.061961 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061693 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-console-config" (OuterVolumeSpecName: "console-config") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:42:40.062022 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.061883 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:42:40.063501 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.063454 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:42:40.063611 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.063510 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:42:40.063611 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.063518 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2afa8750-55df-4d63-865d-74a16e1cbb11-kube-api-access-96kbb" (OuterVolumeSpecName: "kube-api-access-96kbb") pod "2afa8750-55df-4d63-865d-74a16e1cbb11" (UID: "2afa8750-55df-4d63-865d-74a16e1cbb11"). InnerVolumeSpecName "kube-api-access-96kbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:42:40.162512 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162489 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-console-config\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.162512 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162509 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-trusted-ca-bundle\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.162662 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162518 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.162662 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162527 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-oauth-config\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.162662 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162535 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96kbb\" (UniqueName: \"kubernetes.io/projected/2afa8750-55df-4d63-865d-74a16e1cbb11-kube-api-access-96kbb\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.162662 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162547 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2afa8750-55df-4d63-865d-74a16e1cbb11-oauth-serving-cert\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.162662 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.162568 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2afa8750-55df-4d63-865d-74a16e1cbb11-console-serving-cert\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:42:40.747038 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.747006 2578 generic.go:356] "Generic (PLEG): container finished" podID="2afa8750-55df-4d63-865d-74a16e1cbb11" containerID="ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4" exitCode=0 Jul 16 21:42:40.747403 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.747047 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9c6879d9-klztl" Jul 16 21:42:40.747403 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.747097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c6879d9-klztl" event={"ID":"2afa8750-55df-4d63-865d-74a16e1cbb11","Type":"ContainerDied","Data":"ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4"} Jul 16 21:42:40.747403 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.747141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9c6879d9-klztl" event={"ID":"2afa8750-55df-4d63-865d-74a16e1cbb11","Type":"ContainerDied","Data":"21b525ea43366a45c38db21977744e79e11acdcae70a5a2b65099a65f6a098f4"} Jul 16 21:42:40.747403 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.747161 2578 scope.go:117] "RemoveContainer" containerID="ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4" Jul 16 21:42:40.755908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.755891 2578 scope.go:117] "RemoveContainer" containerID="ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4" Jul 16 21:42:40.756160 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:42:40.756138 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4\": container with ID starting with ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4 not found: ID does not exist" containerID="ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4" Jul 16 21:42:40.756202 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.756167 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4"} err="failed to get container status \"ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4\": rpc error: code = NotFound desc = could not find container \"ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4\": container with ID starting with ca23c03b5ad491de42eee894448c0f7187a6bc65d0ed86a8444c7f6e27be01f4 not found: ID does not exist" Jul 16 21:42:40.762914 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.762888 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b9c6879d9-klztl"] Jul 16 21:42:40.767119 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:40.767087 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b9c6879d9-klztl"] Jul 16 21:42:42.674353 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:42:42.674321 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2afa8750-55df-4d63-865d-74a16e1cbb11" path="/var/lib/kubelet/pods/2afa8750-55df-4d63-865d-74a16e1cbb11/volumes" Jul 16 21:43:30.388892 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.388822 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc"] Jul 16 21:43:30.389319 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.389093 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afa8750-55df-4d63-865d-74a16e1cbb11" containerName="console" Jul 16 21:43:30.389319 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.389103 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afa8750-55df-4d63-865d-74a16e1cbb11" containerName="console" Jul 16 21:43:30.389319 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.389162 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afa8750-55df-4d63-865d-74a16e1cbb11" containerName="console" Jul 16 21:43:30.392067 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.392049 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.394486 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.394462 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-82wwz\"" Jul 16 21:43:30.394735 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.394716 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 21:43:30.394839 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.394818 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:43:30.404752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.404731 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc"] Jul 16 21:43:30.516513 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.516483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pbq8\" (UniqueName: \"kubernetes.io/projected/77e1098e-df24-44eb-97f2-56890010bf7f-kube-api-access-6pbq8\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxlvc\" (UID: \"77e1098e-df24-44eb-97f2-56890010bf7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.516636 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.516536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e1098e-df24-44eb-97f2-56890010bf7f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxlvc\" (UID: \"77e1098e-df24-44eb-97f2-56890010bf7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.617571 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.617527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e1098e-df24-44eb-97f2-56890010bf7f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxlvc\" (UID: \"77e1098e-df24-44eb-97f2-56890010bf7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.617693 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.617608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pbq8\" (UniqueName: \"kubernetes.io/projected/77e1098e-df24-44eb-97f2-56890010bf7f-kube-api-access-6pbq8\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxlvc\" (UID: \"77e1098e-df24-44eb-97f2-56890010bf7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.617888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.617870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e1098e-df24-44eb-97f2-56890010bf7f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxlvc\" (UID: \"77e1098e-df24-44eb-97f2-56890010bf7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.625411 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.625389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pbq8\" (UniqueName: \"kubernetes.io/projected/77e1098e-df24-44eb-97f2-56890010bf7f-kube-api-access-6pbq8\") pod \"cert-manager-operator-controller-manager-55c444bf86-bxlvc\" (UID: \"77e1098e-df24-44eb-97f2-56890010bf7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.700958 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.700898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" Jul 16 21:43:30.836892 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.836862 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc"] Jul 16 21:43:30.841111 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:43:30.841087 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e1098e_df24_44eb_97f2_56890010bf7f.slice/crio-687c11819cb40fa88998a673a1610a16c6fbee01b1f995b3284dcf0b09edd1e5 WatchSource:0}: Error finding container 687c11819cb40fa88998a673a1610a16c6fbee01b1f995b3284dcf0b09edd1e5: Status 404 returned error can't find the container with id 687c11819cb40fa88998a673a1610a16c6fbee01b1f995b3284dcf0b09edd1e5 Jul 16 21:43:30.921867 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:30.921839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" event={"ID":"77e1098e-df24-44eb-97f2-56890010bf7f","Type":"ContainerStarted","Data":"687c11819cb40fa88998a673a1610a16c6fbee01b1f995b3284dcf0b09edd1e5"} Jul 16 21:43:33.937939 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:33.937859 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" event={"ID":"77e1098e-df24-44eb-97f2-56890010bf7f","Type":"ContainerStarted","Data":"1dfe07fe479afdaab96281f480d06e0ebc2abf0d25c26356b782df34f9ebfe52"} Jul 16 21:43:33.959277 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:33.959223 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bxlvc" podStartSLOduration=1.266540798 podStartE2EDuration="3.959204547s" totalImagesPullingTime="2.692663749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:43:33.958590209 +0000 UTC m=+279.759476867" watchObservedRunningTime="2026-07-16 21:43:33.959204547 +0000 UTC m=+279.760091200" Jul 16 21:43:45.058259 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.058223 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g28dd"] Jul 16 21:43:45.061668 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.061618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.064111 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.064093 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 21:43:45.064212 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.064115 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bm8ht\"" Jul 16 21:43:45.064814 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.064795 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 21:43:45.069748 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.069729 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g28dd"] Jul 16 21:43:45.119648 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.119623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqhmg\" (UniqueName: \"kubernetes.io/projected/5a0109eb-552a-4b28-a42e-f794541417d1-kube-api-access-xqhmg\") pod \"cert-manager-55b4d954d5-g28dd\" (UID: \"5a0109eb-552a-4b28-a42e-f794541417d1\") " pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.119750 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.119662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a0109eb-552a-4b28-a42e-f794541417d1-bound-sa-token\") pod \"cert-manager-55b4d954d5-g28dd\" (UID: \"5a0109eb-552a-4b28-a42e-f794541417d1\") " pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.220604 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.220573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqhmg\" (UniqueName: \"kubernetes.io/projected/5a0109eb-552a-4b28-a42e-f794541417d1-kube-api-access-xqhmg\") pod \"cert-manager-55b4d954d5-g28dd\" (UID: \"5a0109eb-552a-4b28-a42e-f794541417d1\") " pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.220714 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.220618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a0109eb-552a-4b28-a42e-f794541417d1-bound-sa-token\") pod \"cert-manager-55b4d954d5-g28dd\" (UID: \"5a0109eb-552a-4b28-a42e-f794541417d1\") " pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.228713 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.228680 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a0109eb-552a-4b28-a42e-f794541417d1-bound-sa-token\") pod \"cert-manager-55b4d954d5-g28dd\" (UID: \"5a0109eb-552a-4b28-a42e-f794541417d1\") " pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.228798 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.228741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqhmg\" (UniqueName: \"kubernetes.io/projected/5a0109eb-552a-4b28-a42e-f794541417d1-kube-api-access-xqhmg\") pod \"cert-manager-55b4d954d5-g28dd\" (UID: \"5a0109eb-552a-4b28-a42e-f794541417d1\") " pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.379874 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.379800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g28dd" Jul 16 21:43:45.506223 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.506193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g28dd"] Jul 16 21:43:45.509087 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:43:45.509053 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a0109eb_552a_4b28_a42e_f794541417d1.slice/crio-7b1c3365babcc6052dcd1201e1bae4357fa80bb5dc137917543b12ca7cfeedba WatchSource:0}: Error finding container 7b1c3365babcc6052dcd1201e1bae4357fa80bb5dc137917543b12ca7cfeedba: Status 404 returned error can't find the container with id 7b1c3365babcc6052dcd1201e1bae4357fa80bb5dc137917543b12ca7cfeedba Jul 16 21:43:45.987796 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:45.987757 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g28dd" event={"ID":"5a0109eb-552a-4b28-a42e-f794541417d1","Type":"ContainerStarted","Data":"7b1c3365babcc6052dcd1201e1bae4357fa80bb5dc137917543b12ca7cfeedba"} Jul 16 21:43:49.002818 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:49.002781 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g28dd" event={"ID":"5a0109eb-552a-4b28-a42e-f794541417d1","Type":"ContainerStarted","Data":"1c4d4594b94699e0d310a4df36824b6433a07ce89fe5d7a90ac0419dcc325e8f"} Jul 16 21:43:49.020382 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:49.020335 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-g28dd" podStartSLOduration=1.055486031 podStartE2EDuration="4.02032123s" totalImagesPullingTime="2.964835199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:43:49.018833571 +0000 UTC m=+294.819720221" watchObservedRunningTime="2026-07-16 21:43:49.02032123 +0000 UTC m=+294.821207881" Jul 16 21:43:54.525297 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:54.525271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:43:54.529116 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:54.529095 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:43:54.535579 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:43:54.535541 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 21:44:09.115041 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.115009 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b"] Jul 16 21:44:09.120806 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.120785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.123112 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.123091 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 21:44:09.123218 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.123188 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 21:44:09.123729 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.123714 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 21:44:09.123847 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.123832 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-htzmn\"" Jul 16 21:44:09.124361 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.124347 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 21:44:09.130904 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.130883 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b"] Jul 16 21:44:09.194486 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.194457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c62bb8eb-c617-4c20-a61e-18befaa656e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.194616 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.194495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c62bb8eb-c617-4c20-a61e-18befaa656e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.194616 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.194521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c62bb8eb-c617-4c20-a61e-18befaa656e4-tmp\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.194616 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.194550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c62bb8eb-c617-4c20-a61e-18befaa656e4-manifests\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.194616 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.194613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kb4d\" (UniqueName: \"kubernetes.io/projected/c62bb8eb-c617-4c20-a61e-18befaa656e4-kube-api-access-7kb4d\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.295927 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.295895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c62bb8eb-c617-4c20-a61e-18befaa656e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.295927 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.295929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c62bb8eb-c617-4c20-a61e-18befaa656e4-tmp\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.296075 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.295946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c62bb8eb-c617-4c20-a61e-18befaa656e4-manifests\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.296075 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.295972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kb4d\" (UniqueName: \"kubernetes.io/projected/c62bb8eb-c617-4c20-a61e-18befaa656e4-kube-api-access-7kb4d\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.296075 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.296010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c62bb8eb-c617-4c20-a61e-18befaa656e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.296404 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.296381 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c62bb8eb-c617-4c20-a61e-18befaa656e4-manifests\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.298417 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.298399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c62bb8eb-c617-4c20-a61e-18befaa656e4-tmp\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.298655 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.298639 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c62bb8eb-c617-4c20-a61e-18befaa656e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.298723 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.298668 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c62bb8eb-c617-4c20-a61e-18befaa656e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.305010 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.304982 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kb4d\" (UniqueName: \"kubernetes.io/projected/c62bb8eb-c617-4c20-a61e-18befaa656e4-kube-api-access-7kb4d\") pod \"opendatahub-operator-controller-manager-db5748bcb-h8d6b\" (UID: \"c62bb8eb-c617-4c20-a61e-18befaa656e4\") " pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.431732 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.431677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:09.577303 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.577230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b"] Jul 16 21:44:09.580056 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:44:09.580026 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62bb8eb_c617_4c20_a61e_18befaa656e4.slice/crio-a821916862d11f0d199bf3e283736f41fdc838f4332f31b0476c3d19c2010ae0 WatchSource:0}: Error finding container a821916862d11f0d199bf3e283736f41fdc838f4332f31b0476c3d19c2010ae0: Status 404 returned error can't find the container with id a821916862d11f0d199bf3e283736f41fdc838f4332f31b0476c3d19c2010ae0 Jul 16 21:44:09.582426 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:09.582405 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:44:10.085849 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:10.085819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" event={"ID":"c62bb8eb-c617-4c20-a61e-18befaa656e4","Type":"ContainerStarted","Data":"a821916862d11f0d199bf3e283736f41fdc838f4332f31b0476c3d19c2010ae0"} Jul 16 21:44:13.100255 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:13.100216 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" event={"ID":"c62bb8eb-c617-4c20-a61e-18befaa656e4","Type":"ContainerStarted","Data":"9e814330e34bf536164fea6f6659a1d322f1e277e47501a1b08b92bd8e3b7512"} Jul 16 21:44:14.106005 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:14.105927 2578 generic.go:356] "Generic (PLEG): container finished" podID="c62bb8eb-c617-4c20-a61e-18befaa656e4" containerID="9e814330e34bf536164fea6f6659a1d322f1e277e47501a1b08b92bd8e3b7512" exitCode=0 Jul 16 21:44:14.106355 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:14.106012 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" event={"ID":"c62bb8eb-c617-4c20-a61e-18befaa656e4","Type":"ContainerDied","Data":"9e814330e34bf536164fea6f6659a1d322f1e277e47501a1b08b92bd8e3b7512"} Jul 16 21:44:15.112455 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:15.112416 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" event={"ID":"c62bb8eb-c617-4c20-a61e-18befaa656e4","Type":"ContainerStarted","Data":"3110616a1405982ac97aa8b703106d4bc6d7712441b3a1beaebf869c2da0cac9"} Jul 16 21:44:15.112949 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:15.112608 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:15.141054 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:15.141004 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" podStartSLOduration=1.68602769 podStartE2EDuration="6.140989709s" totalImagesPullingTime="3.378667249s" totalInitContainerRuntime="1.07629477s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:44:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:44:15.138786032 +0000 UTC m=+320.939672682" watchObservedRunningTime="2026-07-16 21:44:15.140989709 +0000 UTC m=+320.941876359" Jul 16 21:44:26.118070 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.118041 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-db5748bcb-h8d6b" Jul 16 21:44:26.456022 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.455950 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj"] Jul 16 21:44:26.462674 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.462657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.465352 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.465333 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 21:44:26.465466 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.465334 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v2t65\"" Jul 16 21:44:26.465466 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.465423 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 21:44:26.465587 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.465490 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:44:26.465587 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.465519 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 21:44:26.465977 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.465960 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 21:44:26.472035 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.472014 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj"] Jul 16 21:44:26.531819 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.531792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9da3aac1-ee2a-4488-a610-f5e328e77804-cert\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.531924 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.531839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9da3aac1-ee2a-4488-a610-f5e328e77804-manager-config\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.531924 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.531892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwzfl\" (UniqueName: \"kubernetes.io/projected/9da3aac1-ee2a-4488-a610-f5e328e77804-kube-api-access-gwzfl\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.532010 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.531926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9da3aac1-ee2a-4488-a610-f5e328e77804-metrics-cert\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.633151 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.633125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9da3aac1-ee2a-4488-a610-f5e328e77804-cert\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.633262 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.633161 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9da3aac1-ee2a-4488-a610-f5e328e77804-manager-config\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.633262 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.633181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwzfl\" (UniqueName: \"kubernetes.io/projected/9da3aac1-ee2a-4488-a610-f5e328e77804-kube-api-access-gwzfl\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.633262 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.633201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9da3aac1-ee2a-4488-a610-f5e328e77804-metrics-cert\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.633812 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.633749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9da3aac1-ee2a-4488-a610-f5e328e77804-manager-config\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.635827 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.635804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9da3aac1-ee2a-4488-a610-f5e328e77804-cert\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.635904 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.635811 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9da3aac1-ee2a-4488-a610-f5e328e77804-metrics-cert\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.643309 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.643287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwzfl\" (UniqueName: \"kubernetes.io/projected/9da3aac1-ee2a-4488-a610-f5e328e77804-kube-api-access-gwzfl\") pod \"lws-controller-manager-6844f9fcbb-4nnbj\" (UID: \"9da3aac1-ee2a-4488-a610-f5e328e77804\") " pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.771738 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.771700 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:26.903438 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:26.903409 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj"] Jul 16 21:44:26.906426 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:44:26.906396 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da3aac1_ee2a_4488_a610_f5e328e77804.slice/crio-4b38340967ae1f1cf0fd3d259d79fd32776903079ceac2ae8f1716dcd3800989 WatchSource:0}: Error finding container 4b38340967ae1f1cf0fd3d259d79fd32776903079ceac2ae8f1716dcd3800989: Status 404 returned error can't find the container with id 4b38340967ae1f1cf0fd3d259d79fd32776903079ceac2ae8f1716dcd3800989 Jul 16 21:44:27.161196 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:27.161124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" event={"ID":"9da3aac1-ee2a-4488-a610-f5e328e77804","Type":"ContainerStarted","Data":"4b38340967ae1f1cf0fd3d259d79fd32776903079ceac2ae8f1716dcd3800989"} Jul 16 21:44:30.176799 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:30.176764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" event={"ID":"9da3aac1-ee2a-4488-a610-f5e328e77804","Type":"ContainerStarted","Data":"d7069a1866b02800e0b7412d5114e939c192a6ef1b72358e64c7176ea5d53db8"} Jul 16 21:44:30.177187 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:30.176847 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:30.241701 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:30.241643 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" podStartSLOduration=1.785503371 podStartE2EDuration="4.241619605s" totalImagesPullingTime="2.456116234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:44:30.239365063 +0000 UTC m=+336.040251734" watchObservedRunningTime="2026-07-16 21:44:30.241619605 +0000 UTC m=+336.042506260" Jul 16 21:44:37.506466 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.506422 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb"] Jul 16 21:44:37.511204 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.511184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.515847 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.515827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 21:44:37.515945 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.515881 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r57rk\"" Jul 16 21:44:37.516092 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.516077 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 21:44:37.516092 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.516084 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 21:44:37.516721 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.516703 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 21:44:37.526443 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.526415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb"] Jul 16 21:44:37.629318 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.629478 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.629478 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25430bde-92fd-4591-a832-cd5df76d33d6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.629478 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfldw\" (UniqueName: \"kubernetes.io/projected/25430bde-92fd-4591-a832-cd5df76d33d6-kube-api-access-zfldw\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.629478 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629445 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25430bde-92fd-4591-a832-cd5df76d33d6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.629478 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629465 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.629724 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.629491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25430bde-92fd-4591-a832-cd5df76d33d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730367 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25430bde-92fd-4591-a832-cd5df76d33d6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730545 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfldw\" (UniqueName: \"kubernetes.io/projected/25430bde-92fd-4591-a832-cd5df76d33d6-kube-api-access-zfldw\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730545 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25430bde-92fd-4591-a832-cd5df76d33d6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730545 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730545 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25430bde-92fd-4591-a832-cd5df76d33d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730545 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.730837 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.730634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.731327 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.731302 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/25430bde-92fd-4591-a832-cd5df76d33d6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.732961 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.732929 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/25430bde-92fd-4591-a832-cd5df76d33d6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.733540 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.733517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.733670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.733527 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.733766 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.733746 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/25430bde-92fd-4591-a832-cd5df76d33d6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.753894 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.753869 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25430bde-92fd-4591-a832-cd5df76d33d6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.755495 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.755468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfldw\" (UniqueName: \"kubernetes.io/projected/25430bde-92fd-4591-a832-cd5df76d33d6-kube-api-access-zfldw\") pod \"istiod-openshift-gateway-94bb8fbfd-czrgb\" (UID: \"25430bde-92fd-4591-a832-cd5df76d33d6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.820773 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.820736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:37.991321 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:37.991289 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb"] Jul 16 21:44:37.994356 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:44:37.994333 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25430bde_92fd_4591_a832_cd5df76d33d6.slice/crio-bba26c27e04da3291cb93f558d20bbb80cbf5521f515610a3a5e504a0c83b3e8 WatchSource:0}: Error finding container bba26c27e04da3291cb93f558d20bbb80cbf5521f515610a3a5e504a0c83b3e8: Status 404 returned error can't find the container with id bba26c27e04da3291cb93f558d20bbb80cbf5521f515610a3a5e504a0c83b3e8 Jul 16 21:44:38.213294 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:38.213221 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" event={"ID":"25430bde-92fd-4591-a832-cd5df76d33d6","Type":"ContainerStarted","Data":"bba26c27e04da3291cb93f558d20bbb80cbf5521f515610a3a5e504a0c83b3e8"} Jul 16 21:44:41.184719 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:41.184686 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6844f9fcbb-4nnbj" Jul 16 21:44:41.700961 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:41.700924 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 16 21:44:41.701044 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:41.700991 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 16 21:44:42.235777 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:42.235739 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" event={"ID":"25430bde-92fd-4591-a832-cd5df76d33d6","Type":"ContainerStarted","Data":"d6e5e2047d5f0bed493d4acb74987b6203c8a603cb09643cba46fd4538ca518c"} Jul 16 21:44:42.236214 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:42.235960 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:44:42.237686 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:42.237664 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-czrgb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 21:44:42.237798 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:42.237718 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" podUID="25430bde-92fd-4591-a832-cd5df76d33d6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:44:42.264308 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:42.264262 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" podStartSLOduration=1.55972255 podStartE2EDuration="5.264249431s" totalImagesPullingTime="3.704526881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:44:42.261364483 +0000 UTC m=+348.062251134" watchObservedRunningTime="2026-07-16 21:44:42.264249431 +0000 UTC m=+348.065136082" Jul 16 21:44:43.240876 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:44:43.240850 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-czrgb" Jul 16 21:46:14.682231 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:14.682195 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd4899969-c8hhp"] Jul 16 21:46:39.707411 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:39.707356 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cd4899969-c8hhp" podUID="1da38d7e-21f1-4cfd-b490-f95a4500d8a7" containerName="console" containerID="cri-o://652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91" gracePeriod=15 Jul 16 21:46:39.946888 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:39.946867 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:46:40.042765 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-serving-cert\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.042923 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042773 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-config\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.042923 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042809 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-trusted-ca-bundle\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.042923 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-service-ca\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.042923 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-oauth-config\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.042923 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042892 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccbk6\" (UniqueName: \"kubernetes.io/projected/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-kube-api-access-ccbk6\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.043164 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.042956 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-oauth-serving-cert\") pod \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\" (UID: \"1da38d7e-21f1-4cfd-b490-f95a4500d8a7\") " Jul 16 21:46:40.043287 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.043259 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-service-ca" (OuterVolumeSpecName: "service-ca") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:46:40.043287 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.043272 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-config" (OuterVolumeSpecName: "console-config") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:46:40.043410 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.043292 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:46:40.043533 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.043511 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:46:40.045021 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.044997 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:46:40.045147 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.045054 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:46:40.045270 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.045255 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-kube-api-access-ccbk6" (OuterVolumeSpecName: "kube-api-access-ccbk6") pod "1da38d7e-21f1-4cfd-b490-f95a4500d8a7" (UID: "1da38d7e-21f1-4cfd-b490-f95a4500d8a7"). InnerVolumeSpecName "kube-api-access-ccbk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:46:40.143919 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143895 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-oauth-serving-cert\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.143919 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143916 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-serving-cert\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.144047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143925 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-config\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.144047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143934 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-trusted-ca-bundle\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.144047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143942 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.144047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143950 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-console-oauth-config\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.144047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.143957 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccbk6\" (UniqueName: \"kubernetes.io/projected/1da38d7e-21f1-4cfd-b490-f95a4500d8a7-kube-api-access-ccbk6\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:46:40.751717 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.751681 2578 generic.go:356] "Generic (PLEG): container finished" podID="1da38d7e-21f1-4cfd-b490-f95a4500d8a7" containerID="652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91" exitCode=0 Jul 16 21:46:40.752054 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.751731 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd4899969-c8hhp" Jul 16 21:46:40.752054 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.751771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd4899969-c8hhp" event={"ID":"1da38d7e-21f1-4cfd-b490-f95a4500d8a7","Type":"ContainerDied","Data":"652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91"} Jul 16 21:46:40.752054 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.751826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd4899969-c8hhp" event={"ID":"1da38d7e-21f1-4cfd-b490-f95a4500d8a7","Type":"ContainerDied","Data":"05bf10e420ec8a07f39a31c34dadc70a7b1e43ffbe465346bd704db5e0c58e32"} Jul 16 21:46:40.752054 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.751850 2578 scope.go:117] "RemoveContainer" containerID="652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91" Jul 16 21:46:40.759921 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.759906 2578 scope.go:117] "RemoveContainer" containerID="652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91" Jul 16 21:46:40.760168 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:46:40.760151 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91\": container with ID starting with 652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91 not found: ID does not exist" containerID="652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91" Jul 16 21:46:40.760220 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.760178 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91"} err="failed to get container status \"652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91\": rpc error: code = NotFound desc = could not find container \"652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91\": container with ID starting with 652e2301b8fa1b035aa467494c3126f66048bb287be9cc881dedae5df2e08b91 not found: ID does not exist" Jul 16 21:46:40.767377 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.767357 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd4899969-c8hhp"] Jul 16 21:46:40.771183 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:40.771165 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cd4899969-c8hhp"] Jul 16 21:46:42.674838 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:46:42.674808 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da38d7e-21f1-4cfd-b490-f95a4500d8a7" path="/var/lib/kubelet/pods/1da38d7e-21f1-4cfd-b490-f95a4500d8a7/volumes" Jul 16 21:47:20.487200 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.487171 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:20.487670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.487445 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da38d7e-21f1-4cfd-b490-f95a4500d8a7" containerName="console" Jul 16 21:47:20.487670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.487455 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da38d7e-21f1-4cfd-b490-f95a4500d8a7" containerName="console" Jul 16 21:47:20.487670 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.487513 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da38d7e-21f1-4cfd-b490-f95a4500d8a7" containerName="console" Jul 16 21:47:20.490367 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.490352 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.492813 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.492790 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 21:47:20.492813 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.492799 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 21:47:20.493668 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.493652 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mq8xk\"" Jul 16 21:47:20.493754 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.493654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 21:47:20.500143 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.500122 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:20.593196 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.593160 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:20.630981 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.630949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d6ab4-f49a-4490-a14e-a7942c663e30-config-file\") pod \"limitador-limitador-64c94fd64f-qxgmb\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.631115 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.631008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csj26\" (UniqueName: \"kubernetes.io/projected/153d6ab4-f49a-4490-a14e-a7942c663e30-kube-api-access-csj26\") pod \"limitador-limitador-64c94fd64f-qxgmb\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.731988 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.731958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csj26\" (UniqueName: \"kubernetes.io/projected/153d6ab4-f49a-4490-a14e-a7942c663e30-kube-api-access-csj26\") pod \"limitador-limitador-64c94fd64f-qxgmb\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.732123 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.732015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d6ab4-f49a-4490-a14e-a7942c663e30-config-file\") pod \"limitador-limitador-64c94fd64f-qxgmb\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.732581 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.732546 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d6ab4-f49a-4490-a14e-a7942c663e30-config-file\") pod \"limitador-limitador-64c94fd64f-qxgmb\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.739872 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.739810 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csj26\" (UniqueName: \"kubernetes.io/projected/153d6ab4-f49a-4490-a14e-a7942c663e30-kube-api-access-csj26\") pod \"limitador-limitador-64c94fd64f-qxgmb\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.801248 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.801225 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:20.933807 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:20.933775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:20.937419 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:47:20.937393 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod153d6ab4_f49a_4490_a14e_a7942c663e30.slice/crio-3afdf0cc1728013402f6b9fb68d9117f7f8595150e99094acaa364f9a1e734c8 WatchSource:0}: Error finding container 3afdf0cc1728013402f6b9fb68d9117f7f8595150e99094acaa364f9a1e734c8: Status 404 returned error can't find the container with id 3afdf0cc1728013402f6b9fb68d9117f7f8595150e99094acaa364f9a1e734c8 Jul 16 21:47:21.919274 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:21.919206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" event={"ID":"153d6ab4-f49a-4490-a14e-a7942c663e30","Type":"ContainerStarted","Data":"3afdf0cc1728013402f6b9fb68d9117f7f8595150e99094acaa364f9a1e734c8"} Jul 16 21:47:23.931366 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:23.931287 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" event={"ID":"153d6ab4-f49a-4490-a14e-a7942c663e30","Type":"ContainerStarted","Data":"d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587"} Jul 16 21:47:23.931732 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:23.931429 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:23.949092 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:23.949045 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" podStartSLOduration=1.213230097 podStartE2EDuration="3.949028533s" totalImagesPullingTime="2.735798436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:23.948511849 +0000 UTC m=+509.749398501" watchObservedRunningTime="2026-07-16 21:47:23.949028533 +0000 UTC m=+509.749915208" Jul 16 21:47:34.936205 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:34.936176 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:35.995795 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:35.995763 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:35.996261 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:35.996053 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" podUID="153d6ab4-f49a-4490-a14e-a7942c663e30" containerName="limitador" containerID="cri-o://d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587" gracePeriod=30 Jul 16 21:47:36.541316 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.541289 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:36.656830 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.656759 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csj26\" (UniqueName: \"kubernetes.io/projected/153d6ab4-f49a-4490-a14e-a7942c663e30-kube-api-access-csj26\") pod \"153d6ab4-f49a-4490-a14e-a7942c663e30\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " Jul 16 21:47:36.656830 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.656822 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d6ab4-f49a-4490-a14e-a7942c663e30-config-file\") pod \"153d6ab4-f49a-4490-a14e-a7942c663e30\" (UID: \"153d6ab4-f49a-4490-a14e-a7942c663e30\") " Jul 16 21:47:36.657184 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.657164 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/153d6ab4-f49a-4490-a14e-a7942c663e30-config-file" (OuterVolumeSpecName: "config-file") pod "153d6ab4-f49a-4490-a14e-a7942c663e30" (UID: "153d6ab4-f49a-4490-a14e-a7942c663e30"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:47:36.659021 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.659004 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/153d6ab4-f49a-4490-a14e-a7942c663e30-kube-api-access-csj26" (OuterVolumeSpecName: "kube-api-access-csj26") pod "153d6ab4-f49a-4490-a14e-a7942c663e30" (UID: "153d6ab4-f49a-4490-a14e-a7942c663e30"). InnerVolumeSpecName "kube-api-access-csj26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:47:36.757926 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.757898 2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/153d6ab4-f49a-4490-a14e-a7942c663e30-config-file\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:47:36.757926 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.757922 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csj26\" (UniqueName: \"kubernetes.io/projected/153d6ab4-f49a-4490-a14e-a7942c663e30-kube-api-access-csj26\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:47:36.985044 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.984964 2578 generic.go:356] "Generic (PLEG): container finished" podID="153d6ab4-f49a-4490-a14e-a7942c663e30" containerID="d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587" exitCode=0 Jul 16 21:47:36.985044 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.985005 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" Jul 16 21:47:36.985044 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.985036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" event={"ID":"153d6ab4-f49a-4490-a14e-a7942c663e30","Type":"ContainerDied","Data":"d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587"} Jul 16 21:47:36.985261 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.985079 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qxgmb" event={"ID":"153d6ab4-f49a-4490-a14e-a7942c663e30","Type":"ContainerDied","Data":"3afdf0cc1728013402f6b9fb68d9117f7f8595150e99094acaa364f9a1e734c8"} Jul 16 21:47:36.985261 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.985106 2578 scope.go:117] "RemoveContainer" containerID="d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587" Jul 16 21:47:36.993239 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.993223 2578 scope.go:117] "RemoveContainer" containerID="d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587" Jul 16 21:47:36.993459 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:47:36.993441 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587\": container with ID starting with d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587 not found: ID does not exist" containerID="d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587" Jul 16 21:47:36.993506 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:36.993468 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587"} err="failed to get container status \"d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587\": rpc error: code = NotFound desc = could not find container \"d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587\": container with ID starting with d35d27b3b09fe08ee00b6c98fb733ba385c13426b0e11ef7d8e36d501a490587 not found: ID does not exist" Jul 16 21:47:37.000627 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:37.000606 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:37.004348 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:37.004329 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qxgmb"] Jul 16 21:47:38.674354 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:38.674309 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="153d6ab4-f49a-4490-a14e-a7942c663e30" path="/var/lib/kubelet/pods/153d6ab4-f49a-4490-a14e-a7942c663e30/volumes" Jul 16 21:47:41.520085 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.520050 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5"] Jul 16 21:47:41.520489 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.520379 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="153d6ab4-f49a-4490-a14e-a7942c663e30" containerName="limitador" Jul 16 21:47:41.520489 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.520394 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="153d6ab4-f49a-4490-a14e-a7942c663e30" containerName="limitador" Jul 16 21:47:41.520489 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.520463 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="153d6ab4-f49a-4490-a14e-a7942c663e30" containerName="limitador" Jul 16 21:47:41.525111 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.525091 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.527730 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.527711 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 16 21:47:41.527844 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.527774 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 21:47:41.527844 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.527774 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fhppx\"" Jul 16 21:47:41.528549 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.528527 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:47:41.533681 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.533659 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5"] Jul 16 21:47:41.693185 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.693157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44b58\" (UniqueName: \"kubernetes.io/projected/9d4e3106-8630-4535-8a94-16b8e2f53871-kube-api-access-44b58\") pod \"postgres-69fb4594cf-7z7x5\" (UID: \"9d4e3106-8630-4535-8a94-16b8e2f53871\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.693331 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.693205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9d4e3106-8630-4535-8a94-16b8e2f53871-data\") pod \"postgres-69fb4594cf-7z7x5\" (UID: \"9d4e3106-8630-4535-8a94-16b8e2f53871\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.794198 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.794129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9d4e3106-8630-4535-8a94-16b8e2f53871-data\") pod \"postgres-69fb4594cf-7z7x5\" (UID: \"9d4e3106-8630-4535-8a94-16b8e2f53871\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.794321 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.794202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44b58\" (UniqueName: \"kubernetes.io/projected/9d4e3106-8630-4535-8a94-16b8e2f53871-kube-api-access-44b58\") pod \"postgres-69fb4594cf-7z7x5\" (UID: \"9d4e3106-8630-4535-8a94-16b8e2f53871\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.794589 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.794545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9d4e3106-8630-4535-8a94-16b8e2f53871-data\") pod \"postgres-69fb4594cf-7z7x5\" (UID: \"9d4e3106-8630-4535-8a94-16b8e2f53871\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.802166 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.802147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44b58\" (UniqueName: \"kubernetes.io/projected/9d4e3106-8630-4535-8a94-16b8e2f53871-kube-api-access-44b58\") pod \"postgres-69fb4594cf-7z7x5\" (UID: \"9d4e3106-8630-4535-8a94-16b8e2f53871\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.837024 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.837002 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:41.968621 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:41.967028 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5"] Jul 16 21:47:42.005813 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:42.005783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" event={"ID":"9d4e3106-8630-4535-8a94-16b8e2f53871","Type":"ContainerStarted","Data":"410aafaca1f20763348b4f0518808929c3c8c6877addf2b71454f203eaa56f6b"} Jul 16 21:47:47.038473 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:47.038430 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" event={"ID":"9d4e3106-8630-4535-8a94-16b8e2f53871","Type":"ContainerStarted","Data":"29ae87ad0ba8abd7141f1c8f110dbe30a430ab0aaf3757ad52d7285c98996389"} Jul 16 21:47:47.038914 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:47.038581 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:47.056894 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:47.056842 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" podStartSLOduration=1.072209236 podStartE2EDuration="6.056823899s" totalImagesPullingTime="4.984614663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:47.055204505 +0000 UTC m=+532.856091156" watchObservedRunningTime="2026-07-16 21:47:47.056823899 +0000 UTC m=+532.857710550" Jul 16 21:47:53.072220 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.072193 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7z7x5" Jul 16 21:47:53.633924 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.633889 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bqc9h"] Jul 16 21:47:53.638755 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.638740 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:53.641230 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.641207 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 21:47:53.641352 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.641210 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x8t5s\"" Jul 16 21:47:53.641352 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.641283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 21:47:53.644211 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.644185 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bqc9h"] Jul 16 21:47:53.677988 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.677961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4gc6\" (UniqueName: \"kubernetes.io/projected/016c1449-9fd9-436f-9949-e2653d2d58d6-kube-api-access-g4gc6\") pod \"authorino-5d4b67c446-bqc9h\" (UID: \"016c1449-9fd9-436f-9949-e2653d2d58d6\") " pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:53.778848 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.778820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4gc6\" (UniqueName: \"kubernetes.io/projected/016c1449-9fd9-436f-9949-e2653d2d58d6-kube-api-access-g4gc6\") pod \"authorino-5d4b67c446-bqc9h\" (UID: \"016c1449-9fd9-436f-9949-e2653d2d58d6\") " pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:53.788813 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.788784 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4gc6\" (UniqueName: \"kubernetes.io/projected/016c1449-9fd9-436f-9949-e2653d2d58d6-kube-api-access-g4gc6\") pod \"authorino-5d4b67c446-bqc9h\" (UID: \"016c1449-9fd9-436f-9949-e2653d2d58d6\") " pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:53.817916 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.817886 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bqc9h"] Jul 16 21:47:53.818112 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.818101 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:53.970119 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:53.970095 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bqc9h"] Jul 16 21:47:53.972902 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:47:53.972875 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod016c1449_9fd9_436f_9949_e2653d2d58d6.slice/crio-8c36a5154d611b769a5aa91165df8804ab46fa4b36288d861c323940324700df WatchSource:0}: Error finding container 8c36a5154d611b769a5aa91165df8804ab46fa4b36288d861c323940324700df: Status 404 returned error can't find the container with id 8c36a5154d611b769a5aa91165df8804ab46fa4b36288d861c323940324700df Jul 16 21:47:54.073418 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:54.073387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" event={"ID":"016c1449-9fd9-436f-9949-e2653d2d58d6","Type":"ContainerStarted","Data":"8c36a5154d611b769a5aa91165df8804ab46fa4b36288d861c323940324700df"} Jul 16 21:47:57.090743 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.090711 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" event={"ID":"016c1449-9fd9-436f-9949-e2653d2d58d6","Type":"ContainerStarted","Data":"9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076"} Jul 16 21:47:57.091166 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.090760 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" podUID="016c1449-9fd9-436f-9949-e2653d2d58d6" containerName="authorino" containerID="cri-o://9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076" gracePeriod=30 Jul 16 21:47:57.107609 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.107544 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" podStartSLOduration=1.177671776 podStartE2EDuration="4.107530209s" totalImagesPullingTime="2.929858433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:57.106021597 +0000 UTC m=+542.906908247" watchObservedRunningTime="2026-07-16 21:47:57.107530209 +0000 UTC m=+542.908416859" Jul 16 21:47:57.378652 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.378627 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:57.405176 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.405154 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4gc6\" (UniqueName: \"kubernetes.io/projected/016c1449-9fd9-436f-9949-e2653d2d58d6-kube-api-access-g4gc6\") pod \"016c1449-9fd9-436f-9949-e2653d2d58d6\" (UID: \"016c1449-9fd9-436f-9949-e2653d2d58d6\") " Jul 16 21:47:57.407291 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.407257 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/016c1449-9fd9-436f-9949-e2653d2d58d6-kube-api-access-g4gc6" (OuterVolumeSpecName: "kube-api-access-g4gc6") pod "016c1449-9fd9-436f-9949-e2653d2d58d6" (UID: "016c1449-9fd9-436f-9949-e2653d2d58d6"). InnerVolumeSpecName "kube-api-access-g4gc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:47:57.506054 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:57.505995 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4gc6\" (UniqueName: \"kubernetes.io/projected/016c1449-9fd9-436f-9949-e2653d2d58d6-kube-api-access-g4gc6\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:47:58.096419 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.096387 2578 generic.go:356] "Generic (PLEG): container finished" podID="016c1449-9fd9-436f-9949-e2653d2d58d6" containerID="9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076" exitCode=0 Jul 16 21:47:58.096881 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.096429 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" Jul 16 21:47:58.096881 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.096428 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" event={"ID":"016c1449-9fd9-436f-9949-e2653d2d58d6","Type":"ContainerDied","Data":"9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076"} Jul 16 21:47:58.096881 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.096477 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bqc9h" event={"ID":"016c1449-9fd9-436f-9949-e2653d2d58d6","Type":"ContainerDied","Data":"8c36a5154d611b769a5aa91165df8804ab46fa4b36288d861c323940324700df"} Jul 16 21:47:58.096881 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.096499 2578 scope.go:117] "RemoveContainer" containerID="9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076" Jul 16 21:47:58.105749 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.105734 2578 scope.go:117] "RemoveContainer" containerID="9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076" Jul 16 21:47:58.105987 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:47:58.105969 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076\": container with ID starting with 9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076 not found: ID does not exist" containerID="9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076" Jul 16 21:47:58.106036 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.105995 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076"} err="failed to get container status \"9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076\": rpc error: code = NotFound desc = could not find container \"9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076\": container with ID starting with 9a7e0ddffc9719ff134236861c7f30ae5eda4a9575cad879db31d5f8d1753076 not found: ID does not exist" Jul 16 21:47:58.118216 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.118195 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bqc9h"] Jul 16 21:47:58.122619 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.122599 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bqc9h"] Jul 16 21:47:58.675591 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:58.675529 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="016c1449-9fd9-436f-9949-e2653d2d58d6" path="/var/lib/kubelet/pods/016c1449-9fd9-436f-9949-e2653d2d58d6/volumes" Jul 16 21:47:59.435941 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.435905 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67cdc7578c-bqbxz"] Jul 16 21:47:59.436376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.436335 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016c1449-9fd9-436f-9949-e2653d2d58d6" containerName="authorino" Jul 16 21:47:59.436376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.436351 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="016c1449-9fd9-436f-9949-e2653d2d58d6" containerName="authorino" Jul 16 21:47:59.436491 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.436456 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="016c1449-9fd9-436f-9949-e2653d2d58d6" containerName="authorino" Jul 16 21:47:59.491143 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.491087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-bqbxz"] Jul 16 21:47:59.491316 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.491185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.493548 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.493518 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 21:47:59.493548 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.493569 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 21:47:59.493763 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.493634 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vsbch\"" Jul 16 21:47:59.522047 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.522024 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmqgp\" (UniqueName: \"kubernetes.io/projected/0f99e028-4031-40f6-8ea7-c6eb11869cc3-kube-api-access-xmqgp\") pod \"maas-controller-67cdc7578c-bqbxz\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.522166 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.522103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f99e028-4031-40f6-8ea7-c6eb11869cc3-webhook-cert\") pod \"maas-controller-67cdc7578c-bqbxz\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.616804 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.616771 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t5fws"] Jul 16 21:47:59.622589 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.622544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f99e028-4031-40f6-8ea7-c6eb11869cc3-webhook-cert\") pod \"maas-controller-67cdc7578c-bqbxz\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.622723 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.622608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmqgp\" (UniqueName: \"kubernetes.io/projected/0f99e028-4031-40f6-8ea7-c6eb11869cc3-kube-api-access-xmqgp\") pod \"maas-controller-67cdc7578c-bqbxz\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.625019 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.625000 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f99e028-4031-40f6-8ea7-c6eb11869cc3-webhook-cert\") pod \"maas-controller-67cdc7578c-bqbxz\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.635021 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.634994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmqgp\" (UniqueName: \"kubernetes.io/projected/0f99e028-4031-40f6-8ea7-c6eb11869cc3-kube-api-access-xmqgp\") pod \"maas-controller-67cdc7578c-bqbxz\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.652983 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.652963 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t5fws"] Jul 16 21:47:59.653077 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.653066 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.723014 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.722946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-webhook-cert\") pod \"maas-controller-5b5ff8896-t5fws\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.723143 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.723024 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx6dl\" (UniqueName: \"kubernetes.io/projected/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-kube-api-access-hx6dl\") pod \"maas-controller-5b5ff8896-t5fws\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.802511 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.802483 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:47:59.824233 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.824210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx6dl\" (UniqueName: \"kubernetes.io/projected/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-kube-api-access-hx6dl\") pod \"maas-controller-5b5ff8896-t5fws\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.824302 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.824268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-webhook-cert\") pod \"maas-controller-5b5ff8896-t5fws\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.826887 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.826864 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-webhook-cert\") pod \"maas-controller-5b5ff8896-t5fws\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.833881 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.833858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx6dl\" (UniqueName: \"kubernetes.io/projected/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-kube-api-access-hx6dl\") pod \"maas-controller-5b5ff8896-t5fws\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:47:59.946620 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.946570 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-bqbxz"] Jul 16 21:47:59.949208 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:47:59.949184 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f99e028_4031_40f6_8ea7_c6eb11869cc3.slice/crio-209ffab74b9927f7ab6e04717f09a445ab668f1d877a898155b4c366cf23415b WatchSource:0}: Error finding container 209ffab74b9927f7ab6e04717f09a445ab668f1d877a898155b4c366cf23415b: Status 404 returned error can't find the container with id 209ffab74b9927f7ab6e04717f09a445ab668f1d877a898155b4c366cf23415b Jul 16 21:47:59.964164 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:47:59.964145 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:48:00.108394 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:00.108347 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" event={"ID":"0f99e028-4031-40f6-8ea7-c6eb11869cc3","Type":"ContainerStarted","Data":"209ffab74b9927f7ab6e04717f09a445ab668f1d877a898155b4c366cf23415b"} Jul 16 21:48:00.126864 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:00.126843 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t5fws"] Jul 16 21:48:00.129836 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:48:00.129173 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a40a408_c2fe_4086_a3cc_6e1e4edf4de8.slice/crio-c708e767f79c5c273b8eff973b8ab3b9bd89795f1579502b5f530fd7a3f14407 WatchSource:0}: Error finding container c708e767f79c5c273b8eff973b8ab3b9bd89795f1579502b5f530fd7a3f14407: Status 404 returned error can't find the container with id c708e767f79c5c273b8eff973b8ab3b9bd89795f1579502b5f530fd7a3f14407 Jul 16 21:48:01.119521 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:01.119483 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5ff8896-t5fws" event={"ID":"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8","Type":"ContainerStarted","Data":"c708e767f79c5c273b8eff973b8ab3b9bd89795f1579502b5f530fd7a3f14407"} Jul 16 21:48:03.130016 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:03.129979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5ff8896-t5fws" event={"ID":"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8","Type":"ContainerStarted","Data":"2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84"} Jul 16 21:48:03.130427 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:03.130055 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:48:03.131464 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:03.131441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" event={"ID":"0f99e028-4031-40f6-8ea7-c6eb11869cc3","Type":"ContainerStarted","Data":"c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9"} Jul 16 21:48:03.131612 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:03.131583 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:48:03.151245 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:03.151193 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b5ff8896-t5fws" podStartSLOduration=2.032832974 podStartE2EDuration="4.151174584s" totalImagesPullingTime="2.11834161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:48:03.149995422 +0000 UTC m=+548.950882072" watchObservedRunningTime="2026-07-16 21:48:03.151174584 +0000 UTC m=+548.952061227" Jul 16 21:48:03.170466 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:03.170423 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" podStartSLOduration=1.878366714 podStartE2EDuration="4.170412398s" totalImagesPullingTime="2.292045684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:48:03.16903277 +0000 UTC m=+548.969919436" watchObservedRunningTime="2026-07-16 21:48:03.170412398 +0000 UTC m=+548.971299049" Jul 16 21:48:14.141359 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.141330 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:48:14.141894 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.141676 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:48:14.201011 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.200982 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-bqbxz"] Jul 16 21:48:14.201193 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.201167 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" podUID="0f99e028-4031-40f6-8ea7-c6eb11869cc3" containerName="manager" containerID="cri-o://c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9" gracePeriod=30 Jul 16 21:48:14.453769 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.453747 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:48:14.536012 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.535986 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f99e028-4031-40f6-8ea7-c6eb11869cc3-webhook-cert\") pod \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " Jul 16 21:48:14.536188 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.536044 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmqgp\" (UniqueName: \"kubernetes.io/projected/0f99e028-4031-40f6-8ea7-c6eb11869cc3-kube-api-access-xmqgp\") pod \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\" (UID: \"0f99e028-4031-40f6-8ea7-c6eb11869cc3\") " Jul 16 21:48:14.538375 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.538337 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f99e028-4031-40f6-8ea7-c6eb11869cc3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0f99e028-4031-40f6-8ea7-c6eb11869cc3" (UID: "0f99e028-4031-40f6-8ea7-c6eb11869cc3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:48:14.538502 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.538380 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f99e028-4031-40f6-8ea7-c6eb11869cc3-kube-api-access-xmqgp" (OuterVolumeSpecName: "kube-api-access-xmqgp") pod "0f99e028-4031-40f6-8ea7-c6eb11869cc3" (UID: "0f99e028-4031-40f6-8ea7-c6eb11869cc3"). InnerVolumeSpecName "kube-api-access-xmqgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:48:14.636842 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.636814 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmqgp\" (UniqueName: \"kubernetes.io/projected/0f99e028-4031-40f6-8ea7-c6eb11869cc3-kube-api-access-xmqgp\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:48:14.636842 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:14.636837 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f99e028-4031-40f6-8ea7-c6eb11869cc3-webhook-cert\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:48:15.184484 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.184454 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f99e028-4031-40f6-8ea7-c6eb11869cc3" containerID="c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9" exitCode=0 Jul 16 21:48:15.184484 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.184488 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" event={"ID":"0f99e028-4031-40f6-8ea7-c6eb11869cc3","Type":"ContainerDied","Data":"c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9"} Jul 16 21:48:15.184985 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.184498 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" Jul 16 21:48:15.184985 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.184526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-bqbxz" event={"ID":"0f99e028-4031-40f6-8ea7-c6eb11869cc3","Type":"ContainerDied","Data":"209ffab74b9927f7ab6e04717f09a445ab668f1d877a898155b4c366cf23415b"} Jul 16 21:48:15.184985 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.184548 2578 scope.go:117] "RemoveContainer" containerID="c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9" Jul 16 21:48:15.193074 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.193054 2578 scope.go:117] "RemoveContainer" containerID="c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9" Jul 16 21:48:15.193329 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:48:15.193311 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9\": container with ID starting with c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9 not found: ID does not exist" containerID="c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9" Jul 16 21:48:15.193377 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.193339 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9"} err="failed to get container status \"c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9\": rpc error: code = NotFound desc = could not find container \"c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9\": container with ID starting with c2a4b02e0af12803515cc4580c1ea7d2e656dbfae6cb76723ec83c892c9bdfe9 not found: ID does not exist" Jul 16 21:48:15.201169 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.201146 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-bqbxz"] Jul 16 21:48:15.205791 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:15.205771 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-bqbxz"] Jul 16 21:48:16.675311 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:16.675278 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f99e028-4031-40f6-8ea7-c6eb11869cc3" path="/var/lib/kubelet/pods/0f99e028-4031-40f6-8ea7-c6eb11869cc3/volumes" Jul 16 21:48:35.596714 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.596682 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm"] Jul 16 21:48:35.597127 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.596989 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f99e028-4031-40f6-8ea7-c6eb11869cc3" containerName="manager" Jul 16 21:48:35.597127 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.597000 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f99e028-4031-40f6-8ea7-c6eb11869cc3" containerName="manager" Jul 16 21:48:35.597127 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.597056 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f99e028-4031-40f6-8ea7-c6eb11869cc3" containerName="manager" Jul 16 21:48:35.604782 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.604760 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.609077 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.609056 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b9n4q\"" Jul 16 21:48:35.609974 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.609958 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 21:48:35.657164 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.657141 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm"] Jul 16 21:48:35.696631 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.696602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff45b9-2544-48a0-89ce-624c6cda6c29-openshift-service-ca\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.696743 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.696696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg2xv\" (UniqueName: \"kubernetes.io/projected/ddff45b9-2544-48a0-89ce-624c6cda6c29-kube-api-access-gg2xv\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.696783 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.696741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddff45b9-2544-48a0-89ce-624c6cda6c29-maas-api-tls\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.797602 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.797576 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddff45b9-2544-48a0-89ce-624c6cda6c29-maas-api-tls\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.797726 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.797623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff45b9-2544-48a0-89ce-624c6cda6c29-openshift-service-ca\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.797832 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.797726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg2xv\" (UniqueName: \"kubernetes.io/projected/ddff45b9-2544-48a0-89ce-624c6cda6c29-kube-api-access-gg2xv\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.798281 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.798264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff45b9-2544-48a0-89ce-624c6cda6c29-openshift-service-ca\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.800141 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.800126 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddff45b9-2544-48a0-89ce-624c6cda6c29-maas-api-tls\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.819249 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.819231 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg2xv\" (UniqueName: \"kubernetes.io/projected/ddff45b9-2544-48a0-89ce-624c6cda6c29-kube-api-access-gg2xv\") pod \"maas-api-68fc5bb9d9-fkxsm\" (UID: \"ddff45b9-2544-48a0-89ce-624c6cda6c29\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:35.915106 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:35.915050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:36.062406 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.062379 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm"] Jul 16 21:48:36.064299 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:48:36.064274 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddff45b9_2544_48a0_89ce_624c6cda6c29.slice/crio-3fbdee347e552eda501eef2f0b7898c369c5c2331dc3767d49168514e213249e WatchSource:0}: Error finding container 3fbdee347e552eda501eef2f0b7898c369c5c2331dc3767d49168514e213249e: Status 404 returned error can't find the container with id 3fbdee347e552eda501eef2f0b7898c369c5c2331dc3767d49168514e213249e Jul 16 21:48:36.273227 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.273195 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" event={"ID":"ddff45b9-2544-48a0-89ce-624c6cda6c29","Type":"ContainerStarted","Data":"3fbdee347e552eda501eef2f0b7898c369c5c2331dc3767d49168514e213249e"} Jul 16 21:48:36.491411 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.491377 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-2vrvh"] Jul 16 21:48:36.496424 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.496393 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.508129 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.508107 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g4pjt\"" Jul 16 21:48:36.508129 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.508124 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 21:48:36.519101 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.519081 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-2vrvh"] Jul 16 21:48:36.603908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.603829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77mj5\" (UniqueName: \"kubernetes.io/projected/4de09e9c-9c6b-4a95-90ca-93d524f9c519-kube-api-access-77mj5\") pod \"payload-pre-processing-7964457696-2vrvh\" (UID: \"4de09e9c-9c6b-4a95-90ca-93d524f9c519\") " pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.603908 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.603884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4de09e9c-9c6b-4a95-90ca-93d524f9c519-plugins-config-volume\") pod \"payload-pre-processing-7964457696-2vrvh\" (UID: \"4de09e9c-9c6b-4a95-90ca-93d524f9c519\") " pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.705108 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.705077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77mj5\" (UniqueName: \"kubernetes.io/projected/4de09e9c-9c6b-4a95-90ca-93d524f9c519-kube-api-access-77mj5\") pod \"payload-pre-processing-7964457696-2vrvh\" (UID: \"4de09e9c-9c6b-4a95-90ca-93d524f9c519\") " pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.705254 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.705115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4de09e9c-9c6b-4a95-90ca-93d524f9c519-plugins-config-volume\") pod \"payload-pre-processing-7964457696-2vrvh\" (UID: \"4de09e9c-9c6b-4a95-90ca-93d524f9c519\") " pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.705840 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.705823 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4de09e9c-9c6b-4a95-90ca-93d524f9c519-plugins-config-volume\") pod \"payload-pre-processing-7964457696-2vrvh\" (UID: \"4de09e9c-9c6b-4a95-90ca-93d524f9c519\") " pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.717327 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.717297 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77mj5\" (UniqueName: \"kubernetes.io/projected/4de09e9c-9c6b-4a95-90ca-93d524f9c519-kube-api-access-77mj5\") pod \"payload-pre-processing-7964457696-2vrvh\" (UID: \"4de09e9c-9c6b-4a95-90ca-93d524f9c519\") " pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.806355 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.806328 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:36.962610 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:36.962575 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-2vrvh"] Jul 16 21:48:36.967174 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:48:36.967148 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de09e9c_9c6b_4a95_90ca_93d524f9c519.slice/crio-0bb39a3f657e2d8f111a730683b834e4f1f26ae51f21af626e9b75fc63a26d9d WatchSource:0}: Error finding container 0bb39a3f657e2d8f111a730683b834e4f1f26ae51f21af626e9b75fc63a26d9d: Status 404 returned error can't find the container with id 0bb39a3f657e2d8f111a730683b834e4f1f26ae51f21af626e9b75fc63a26d9d Jul 16 21:48:37.280435 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:37.280396 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" event={"ID":"4de09e9c-9c6b-4a95-90ca-93d524f9c519","Type":"ContainerStarted","Data":"0bb39a3f657e2d8f111a730683b834e4f1f26ae51f21af626e9b75fc63a26d9d"} Jul 16 21:48:39.292196 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:39.292155 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" event={"ID":"4de09e9c-9c6b-4a95-90ca-93d524f9c519","Type":"ContainerStarted","Data":"6c77cbc32fa5ba0242e139e03c10e7f0871ce63d7313de316a8c4f4f41c315dd"} Jul 16 21:48:39.292695 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:39.292224 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:39.293696 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:39.293675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" event={"ID":"ddff45b9-2544-48a0-89ce-624c6cda6c29","Type":"ContainerStarted","Data":"9e8e4d7833b943bb98f0f688b30bd6532cc37eed46c49b0d6936513acb22dec9"} Jul 16 21:48:39.293796 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:39.293763 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:39.310008 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:39.309965 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" podStartSLOduration=1.859645718 podStartE2EDuration="3.309951672s" totalImagesPullingTime="1.450305954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:48:39.308964492 +0000 UTC m=+585.109851162" watchObservedRunningTime="2026-07-16 21:48:39.309951672 +0000 UTC m=+585.110838323" Jul 16 21:48:39.326503 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:39.326455 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" podStartSLOduration=1.974301013 podStartE2EDuration="4.326443139s" totalImagesPullingTime="2.352142126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:48:39.32512265 +0000 UTC m=+585.126009302" watchObservedRunningTime="2026-07-16 21:48:39.326443139 +0000 UTC m=+585.127329794" Jul 16 21:48:45.302737 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:45.302710 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-fkxsm" Jul 16 21:48:50.300299 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:50.300271 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-2vrvh" Jul 16 21:48:52.288574 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.288518 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks"] Jul 16 21:48:52.292064 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.292038 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.294851 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.294827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 16 21:48:52.294974 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.294877 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 21:48:52.295739 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.295720 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9l2xf\"" Jul 16 21:48:52.295837 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.295739 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 21:48:52.306422 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.306401 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks"] Jul 16 21:48:52.428337 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.428309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.428337 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.428339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.428504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.428368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.428504 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.428467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kqv2\" (UniqueName: \"kubernetes.io/projected/b4176a13-a19f-4659-ae18-128b50872e38-kube-api-access-7kqv2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.428593 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.428507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4176a13-a19f-4659-ae18-128b50872e38-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.428593 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.428532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.529608 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.529752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kqv2\" (UniqueName: \"kubernetes.io/projected/b4176a13-a19f-4659-ae18-128b50872e38-kube-api-access-7kqv2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.529752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4176a13-a19f-4659-ae18-128b50872e38-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.529752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.529752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.529752 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529731 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.530009 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.529953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.530074 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.530050 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.530108 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.530075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.532228 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.532207 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4176a13-a19f-4659-ae18-128b50872e38-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.532336 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.532319 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4176a13-a19f-4659-ae18-128b50872e38-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.538896 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.538840 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kqv2\" (UniqueName: \"kubernetes.io/projected/b4176a13-a19f-4659-ae18-128b50872e38-kube-api-access-7kqv2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks\" (UID: \"b4176a13-a19f-4659-ae18-128b50872e38\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.601906 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.601880 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:48:52.738967 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:52.738939 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks"] Jul 16 21:48:52.740983 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:48:52.740956 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4176a13_a19f_4659_ae18_128b50872e38.slice/crio-30846f840ff045f397f58b1d85a69a8e4dc28730a63c3a6effd838886cf21575 WatchSource:0}: Error finding container 30846f840ff045f397f58b1d85a69a8e4dc28730a63c3a6effd838886cf21575: Status 404 returned error can't find the container with id 30846f840ff045f397f58b1d85a69a8e4dc28730a63c3a6effd838886cf21575 Jul 16 21:48:53.359747 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:53.359709 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" event={"ID":"b4176a13-a19f-4659-ae18-128b50872e38","Type":"ContainerStarted","Data":"30846f840ff045f397f58b1d85a69a8e4dc28730a63c3a6effd838886cf21575"} Jul 16 21:48:54.366535 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:54.366499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" event={"ID":"b4176a13-a19f-4659-ae18-128b50872e38","Type":"ContainerStarted","Data":"57ee2099fb78db4080b3a3d7eb77f9662f4595059d9e0c357db23c9f3519de65"} Jul 16 21:48:54.387175 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:54.387125 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" podStartSLOduration=1.206354684 podStartE2EDuration="2.387111433s" totalImagesPullingTime="1.180756749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:48:54.385061809 +0000 UTC m=+600.185948459" watchObservedRunningTime="2026-07-16 21:48:54.387111433 +0000 UTC m=+600.187998086" Jul 16 21:48:54.557011 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:54.556976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:48:54.557219 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:48:54.557203 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:49:02.003120 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.003089 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv"] Jul 16 21:49:02.007889 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.007865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.008669 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.008646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.008790 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.008680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.008790 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.008716 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg5r6\" (UniqueName: \"kubernetes.io/projected/77fc7706-fcc6-4944-a37b-45c02c0884fb-kube-api-access-tg5r6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.008866 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.008773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.008866 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.008812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77fc7706-fcc6-4944-a37b-45c02c0884fb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.008866 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.008843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.010923 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.010901 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 21:49:02.019397 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.019372 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv"] Jul 16 21:49:02.109270 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg5r6\" (UniqueName: \"kubernetes.io/projected/77fc7706-fcc6-4944-a37b-45c02c0884fb-kube-api-access-tg5r6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109423 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77fc7706-fcc6-4944-a37b-45c02c0884fb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109548 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109548 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109548 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109732 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109717 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109862 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.109979 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.109870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.111753 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.111725 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77fc7706-fcc6-4944-a37b-45c02c0884fb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.112034 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.112015 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77fc7706-fcc6-4944-a37b-45c02c0884fb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.119455 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.119431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg5r6\" (UniqueName: \"kubernetes.io/projected/77fc7706-fcc6-4944-a37b-45c02c0884fb-kube-api-access-tg5r6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z68wv\" (UID: \"77fc7706-fcc6-4944-a37b-45c02c0884fb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.317809 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.317782 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:02.455114 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.455055 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv"] Jul 16 21:49:02.457336 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:49:02.457311 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77fc7706_fcc6_4944_a37b_45c02c0884fb.slice/crio-3511ed1d61fe8b26d90f262f249e42441a4dc69dbe83f4664566de612101d31c WatchSource:0}: Error finding container 3511ed1d61fe8b26d90f262f249e42441a4dc69dbe83f4664566de612101d31c: Status 404 returned error can't find the container with id 3511ed1d61fe8b26d90f262f249e42441a4dc69dbe83f4664566de612101d31c Jul 16 21:49:02.602956 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.602895 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:49:02.602956 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.602951 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:49:02.614885 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:02.614865 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:49:03.408529 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:03.408495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" event={"ID":"77fc7706-fcc6-4944-a37b-45c02c0884fb","Type":"ContainerStarted","Data":"d703fb50f50792f628bd2aee370ce72e6b8152ec6c5cdf3caf5a2fb5bece2d2a"} Jul 16 21:49:03.408940 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:03.408577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" event={"ID":"77fc7706-fcc6-4944-a37b-45c02c0884fb","Type":"ContainerStarted","Data":"3511ed1d61fe8b26d90f262f249e42441a4dc69dbe83f4664566de612101d31c"} Jul 16 21:49:03.419765 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:03.419745 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks" Jul 16 21:49:03.450295 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:03.450253 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" podStartSLOduration=2.223429122 podStartE2EDuration="2.450237918s" totalImagesPullingTime="226.808796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:03.448141756 +0000 UTC m=+609.249028407" watchObservedRunningTime="2026-07-16 21:49:03.450237918 +0000 UTC m=+609.251124568" Jul 16 21:49:12.318050 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:12.318013 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:12.321170 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:12.318079 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:12.330454 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:12.330436 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:49:12.459664 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:49:12.459635 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z68wv" Jul 16 21:51:11.004909 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.004836 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d49c6f95d-wsv56"] Jul 16 21:51:11.008220 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.008202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.011648 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.011625 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 21:51:11.011747 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.011625 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x8t5s\"" Jul 16 21:51:11.011747 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.011664 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 16 21:51:11.011747 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.011626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 21:51:11.017310 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.017284 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d49c6f95d-wsv56"] Jul 16 21:51:11.107587 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.107537 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b330e71f-4a8d-448e-8cfa-03baf846f2a9-tls-cert\") pod \"authorino-6d49c6f95d-wsv56\" (UID: \"b330e71f-4a8d-448e-8cfa-03baf846f2a9\") " pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.107701 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.107648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lql88\" (UniqueName: \"kubernetes.io/projected/b330e71f-4a8d-448e-8cfa-03baf846f2a9-kube-api-access-lql88\") pod \"authorino-6d49c6f95d-wsv56\" (UID: \"b330e71f-4a8d-448e-8cfa-03baf846f2a9\") " pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.208422 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.208397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b330e71f-4a8d-448e-8cfa-03baf846f2a9-tls-cert\") pod \"authorino-6d49c6f95d-wsv56\" (UID: \"b330e71f-4a8d-448e-8cfa-03baf846f2a9\") " pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.208530 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.208437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lql88\" (UniqueName: \"kubernetes.io/projected/b330e71f-4a8d-448e-8cfa-03baf846f2a9-kube-api-access-lql88\") pod \"authorino-6d49c6f95d-wsv56\" (UID: \"b330e71f-4a8d-448e-8cfa-03baf846f2a9\") " pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.210876 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.210856 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b330e71f-4a8d-448e-8cfa-03baf846f2a9-tls-cert\") pod \"authorino-6d49c6f95d-wsv56\" (UID: \"b330e71f-4a8d-448e-8cfa-03baf846f2a9\") " pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.218001 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.217977 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lql88\" (UniqueName: \"kubernetes.io/projected/b330e71f-4a8d-448e-8cfa-03baf846f2a9-kube-api-access-lql88\") pod \"authorino-6d49c6f95d-wsv56\" (UID: \"b330e71f-4a8d-448e-8cfa-03baf846f2a9\") " pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.317664 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.317642 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d49c6f95d-wsv56" Jul 16 21:51:11.456800 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.456779 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d49c6f95d-wsv56"] Jul 16 21:51:11.461046 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:51:11.461009 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb330e71f_4a8d_448e_8cfa_03baf846f2a9.slice/crio-314a9f36716931d41277ef1083c4bb814246fb57a22802022712bc1860cd4089 WatchSource:0}: Error finding container 314a9f36716931d41277ef1083c4bb814246fb57a22802022712bc1860cd4089: Status 404 returned error can't find the container with id 314a9f36716931d41277ef1083c4bb814246fb57a22802022712bc1860cd4089 Jul 16 21:51:11.462545 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.462525 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:51:11.973867 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:11.973842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d49c6f95d-wsv56" event={"ID":"b330e71f-4a8d-448e-8cfa-03baf846f2a9","Type":"ContainerStarted","Data":"314a9f36716931d41277ef1083c4bb814246fb57a22802022712bc1860cd4089"} Jul 16 21:51:12.981163 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:12.981124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d49c6f95d-wsv56" event={"ID":"b330e71f-4a8d-448e-8cfa-03baf846f2a9","Type":"ContainerStarted","Data":"fc3bee2c9d6332f78d86684ec0d5ed5147b3be3ace1164cb47d36c95392ddb27"} Jul 16 21:51:13.023092 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:51:13.023034 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d49c6f95d-wsv56" podStartSLOduration=2.560667065 podStartE2EDuration="3.023014682s" totalImagesPullingTime="462.347617ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:13.021884225 +0000 UTC m=+738.822770877" watchObservedRunningTime="2026-07-16 21:51:13.023014682 +0000 UTC m=+738.823901333" Jul 16 21:52:22.940870 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:22.940771 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t5fws"] Jul 16 21:52:22.941394 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:22.941015 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b5ff8896-t5fws" podUID="0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" containerName="manager" containerID="cri-o://2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84" gracePeriod=30 Jul 16 21:52:23.190829 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.190802 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:52:23.197763 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.197713 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx6dl\" (UniqueName: \"kubernetes.io/projected/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-kube-api-access-hx6dl\") pod \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " Jul 16 21:52:23.197885 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.197779 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-webhook-cert\") pod \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\" (UID: \"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8\") " Jul 16 21:52:23.199796 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.199773 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" (UID: "0a40a408-c2fe-4086-a3cc-6e1e4edf4de8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:52:23.199909 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.199834 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-kube-api-access-hx6dl" (OuterVolumeSpecName: "kube-api-access-hx6dl") pod "0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" (UID: "0a40a408-c2fe-4086-a3cc-6e1e4edf4de8"). InnerVolumeSpecName "kube-api-access-hx6dl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:52:23.298378 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.298348 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-webhook-cert\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:52:23.298378 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.298372 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx6dl\" (UniqueName: \"kubernetes.io/projected/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8-kube-api-access-hx6dl\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 21:52:23.306258 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.306233 2578 generic.go:356] "Generic (PLEG): container finished" podID="0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" containerID="2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84" exitCode=0 Jul 16 21:52:23.306360 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.306276 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5ff8896-t5fws" Jul 16 21:52:23.306360 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.306314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5ff8896-t5fws" event={"ID":"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8","Type":"ContainerDied","Data":"2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84"} Jul 16 21:52:23.306453 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.306366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5ff8896-t5fws" event={"ID":"0a40a408-c2fe-4086-a3cc-6e1e4edf4de8","Type":"ContainerDied","Data":"c708e767f79c5c273b8eff973b8ab3b9bd89795f1579502b5f530fd7a3f14407"} Jul 16 21:52:23.306453 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.306388 2578 scope.go:117] "RemoveContainer" containerID="2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84" Jul 16 21:52:23.315348 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.315334 2578 scope.go:117] "RemoveContainer" containerID="2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84" Jul 16 21:52:23.315606 ip-10-0-136-44 kubenswrapper[2578]: E0716 21:52:23.315577 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84\": container with ID starting with 2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84 not found: ID does not exist" containerID="2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84" Jul 16 21:52:23.315703 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.315612 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84"} err="failed to get container status \"2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84\": rpc error: code = NotFound desc = could not find container \"2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84\": container with ID starting with 2d746c3c5156dc778075817a4e2f03be9355d84898590b3dc432d3728279dd84 not found: ID does not exist" Jul 16 21:52:23.327135 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.327114 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t5fws"] Jul 16 21:52:23.336373 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:23.331371 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t5fws"] Jul 16 21:52:24.078322 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.078283 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t9kx4"] Jul 16 21:52:24.078865 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.078846 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" containerName="manager" Jul 16 21:52:24.078945 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.078869 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" containerName="manager" Jul 16 21:52:24.078995 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.078961 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" containerName="manager" Jul 16 21:52:24.083185 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.083164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.086406 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.086383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 21:52:24.086544 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.086383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vsbch\"" Jul 16 21:52:24.086544 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.086422 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 21:52:24.092700 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.092675 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t9kx4"] Jul 16 21:52:24.104133 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.104110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9j74\" (UniqueName: \"kubernetes.io/projected/85271990-63ad-4be6-92fe-e4a41a5bdd4d-kube-api-access-c9j74\") pod \"maas-controller-5b5ff8896-t9kx4\" (UID: \"85271990-63ad-4be6-92fe-e4a41a5bdd4d\") " pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.104240 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.104161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85271990-63ad-4be6-92fe-e4a41a5bdd4d-webhook-cert\") pod \"maas-controller-5b5ff8896-t9kx4\" (UID: \"85271990-63ad-4be6-92fe-e4a41a5bdd4d\") " pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.205425 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.205393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9j74\" (UniqueName: \"kubernetes.io/projected/85271990-63ad-4be6-92fe-e4a41a5bdd4d-kube-api-access-c9j74\") pod \"maas-controller-5b5ff8896-t9kx4\" (UID: \"85271990-63ad-4be6-92fe-e4a41a5bdd4d\") " pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.205607 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.205475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85271990-63ad-4be6-92fe-e4a41a5bdd4d-webhook-cert\") pod \"maas-controller-5b5ff8896-t9kx4\" (UID: \"85271990-63ad-4be6-92fe-e4a41a5bdd4d\") " pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.208091 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.208056 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85271990-63ad-4be6-92fe-e4a41a5bdd4d-webhook-cert\") pod \"maas-controller-5b5ff8896-t9kx4\" (UID: \"85271990-63ad-4be6-92fe-e4a41a5bdd4d\") " pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.214826 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.214802 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9j74\" (UniqueName: \"kubernetes.io/projected/85271990-63ad-4be6-92fe-e4a41a5bdd4d-kube-api-access-c9j74\") pod \"maas-controller-5b5ff8896-t9kx4\" (UID: \"85271990-63ad-4be6-92fe-e4a41a5bdd4d\") " pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.394722 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.394632 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:24.530930 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.530902 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5ff8896-t9kx4"] Jul 16 21:52:24.534231 ip-10-0-136-44 kubenswrapper[2578]: W0716 21:52:24.534206 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85271990_63ad_4be6_92fe_e4a41a5bdd4d.slice/crio-e27660ded2610c15f865abc58c8cfe3069c8a16c0442ffde64a2fe6b1ff565f3 WatchSource:0}: Error finding container e27660ded2610c15f865abc58c8cfe3069c8a16c0442ffde64a2fe6b1ff565f3: Status 404 returned error can't find the container with id e27660ded2610c15f865abc58c8cfe3069c8a16c0442ffde64a2fe6b1ff565f3 Jul 16 21:52:24.675350 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:24.675278 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a40a408-c2fe-4086-a3cc-6e1e4edf4de8" path="/var/lib/kubelet/pods/0a40a408-c2fe-4086-a3cc-6e1e4edf4de8/volumes" Jul 16 21:52:25.319376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:25.319298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" event={"ID":"85271990-63ad-4be6-92fe-e4a41a5bdd4d","Type":"ContainerStarted","Data":"c1183cf80a2f18d0f2a668a3085afa7ce2be36af4f2fe53ea81f6d0dd3829d60"} Jul 16 21:52:25.319376 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:25.319355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" event={"ID":"85271990-63ad-4be6-92fe-e4a41a5bdd4d","Type":"ContainerStarted","Data":"e27660ded2610c15f865abc58c8cfe3069c8a16c0442ffde64a2fe6b1ff565f3"} Jul 16 21:52:25.319863 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:25.319430 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:52:25.336897 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:25.336853 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" podStartSLOduration=0.839158344 podStartE2EDuration="1.336840744s" totalImagesPullingTime="497.6824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:25.335684435 +0000 UTC m=+811.136571086" watchObservedRunningTime="2026-07-16 21:52:25.336840744 +0000 UTC m=+811.137727395" Jul 16 21:52:36.329610 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:52:36.329579 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b5ff8896-t9kx4" Jul 16 21:53:54.585103 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:53:54.585013 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:53:54.586057 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:53:54.586031 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:58:54.616385 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:58:54.616346 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 21:58:54.619231 ip-10-0-136-44 kubenswrapper[2578]: I0716 21:58:54.619205 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:00:00.150923 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.150836 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx"] Jul 16 22:00:00.154375 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.154354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" Jul 16 22:00:00.156954 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.156932 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7tspb\"" Jul 16 22:00:00.162537 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.162510 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx"] Jul 16 22:00:00.167584 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.167537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" Jul 16 22:00:00.307870 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.307847 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx"] Jul 16 22:00:00.310686 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:00:00.310659 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e052d4_ca02_4d03_a330_7bf8a017b4be.slice/crio-ccb1cee24ebbfee3fc2b65c53ea43ec73229025590d959120ea72150b5c7c963 WatchSource:0}: Error finding container ccb1cee24ebbfee3fc2b65c53ea43ec73229025590d959120ea72150b5c7c963: Status 404 returned error can't find the container with id ccb1cee24ebbfee3fc2b65c53ea43ec73229025590d959120ea72150b5c7c963 Jul 16 22:00:00.313835 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.313818 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 22:00:00.432857 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:00.432781 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" event={"ID":"a2e052d4-ca02-4d03-a330-7bf8a017b4be","Type":"ContainerStarted","Data":"ccb1cee24ebbfee3fc2b65c53ea43ec73229025590d959120ea72150b5c7c963"} Jul 16 22:00:05.457548 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:05.457472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" event={"ID":"a2e052d4-ca02-4d03-a330-7bf8a017b4be","Type":"ContainerStarted","Data":"f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e"} Jul 16 22:00:05.472766 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:00:05.472722 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" podStartSLOduration=1.013292957 podStartE2EDuration="5.472705163s" totalImagesPullingTime="4.459412206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:00:05.470941606 +0000 UTC m=+1271.271828257" watchObservedRunningTime="2026-07-16 22:00:05.472705163 +0000 UTC m=+1271.273591816" Jul 16 22:01:00.010463 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:00.010424 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx"] Jul 16 22:01:00.010994 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:00.010688 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" podUID="a2e052d4-ca02-4d03-a330-7bf8a017b4be" containerName="cleanup" containerID="cri-o://f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e" gracePeriod=30 Jul 16 22:01:30.201588 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.201549 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" Jul 16 22:01:30.837778 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.837748 2578 generic.go:356] "Generic (PLEG): container finished" podID="a2e052d4-ca02-4d03-a330-7bf8a017b4be" containerID="f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e" exitCode=137 Jul 16 22:01:30.837936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.837789 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" Jul 16 22:01:30.837936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.837825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" event={"ID":"a2e052d4-ca02-4d03-a330-7bf8a017b4be","Type":"ContainerDied","Data":"f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e"} Jul 16 22:01:30.837936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.837871 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx" event={"ID":"a2e052d4-ca02-4d03-a330-7bf8a017b4be","Type":"ContainerDied","Data":"ccb1cee24ebbfee3fc2b65c53ea43ec73229025590d959120ea72150b5c7c963"} Jul 16 22:01:30.837936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.837894 2578 scope.go:117] "RemoveContainer" containerID="f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e" Jul 16 22:01:30.846328 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.846309 2578 scope.go:117] "RemoveContainer" containerID="f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e" Jul 16 22:01:30.846624 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:01:30.846603 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e\": container with ID starting with f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e not found: ID does not exist" containerID="f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e" Jul 16 22:01:30.846691 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.846638 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e"} err="failed to get container status \"f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e\": rpc error: code = NotFound desc = could not find container \"f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e\": container with ID starting with f25d613deebd5b26d29fc631abb47c3c33669d1cca027d70c8155107463cd60e not found: ID does not exist" Jul 16 22:01:30.853887 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.853864 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx"] Jul 16 22:01:30.856343 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:30.856325 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-hfgxx"] Jul 16 22:01:32.675456 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:01:32.675426 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e052d4-ca02-4d03-a330-7bf8a017b4be" path="/var/lib/kubelet/pods/a2e052d4-ca02-4d03-a330-7bf8a017b4be/volumes" Jul 16 22:03:54.642656 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:03:54.642631 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:03:54.646681 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:03:54.646656 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:04:52.733252 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.733220 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s"] Jul 16 22:04:52.733663 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.733588 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e052d4-ca02-4d03-a330-7bf8a017b4be" containerName="cleanup" Jul 16 22:04:52.733663 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.733603 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e052d4-ca02-4d03-a330-7bf8a017b4be" containerName="cleanup" Jul 16 22:04:52.733738 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.733680 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e052d4-ca02-4d03-a330-7bf8a017b4be" containerName="cleanup" Jul 16 22:04:52.736696 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.736677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.739657 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.739635 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-997f8b52-serving-cert\"" Jul 16 22:04:52.747932 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.747912 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s"] Jul 16 22:04:52.792999 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.792973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppr6k\" (UniqueName: \"kubernetes.io/projected/f2304bd7-28dc-4dbc-9f82-4132cee58617-kube-api-access-ppr6k\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.793137 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.793015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2304bd7-28dc-4dbc-9f82-4132cee58617-openshift-service-ca\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.793137 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.793049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.893566 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.893531 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppr6k\" (UniqueName: \"kubernetes.io/projected/f2304bd7-28dc-4dbc-9f82-4132cee58617-kube-api-access-ppr6k\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.893730 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.893635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2304bd7-28dc-4dbc-9f82-4132cee58617-openshift-service-ca\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.893730 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.893676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.893848 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:04:52.893804 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-serving-cert: secret "maas-api-e2e-ait-997f8b52-serving-cert" not found Jul 16 22:04:52.893900 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:04:52.893887 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls podName:f2304bd7-28dc-4dbc-9f82-4132cee58617 nodeName:}" failed. No retries permitted until 2026-07-16 22:04:53.393862715 +0000 UTC m=+1559.194749364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls") pod "maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" (UID: "f2304bd7-28dc-4dbc-9f82-4132cee58617") : secret "maas-api-e2e-ait-997f8b52-serving-cert" not found Jul 16 22:04:52.894260 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.894237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2304bd7-28dc-4dbc-9f82-4132cee58617-openshift-service-ca\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:52.906648 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:52.906625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppr6k\" (UniqueName: \"kubernetes.io/projected/f2304bd7-28dc-4dbc-9f82-4132cee58617-kube-api-access-ppr6k\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:53.398224 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:53.398190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:53.400876 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:53.400842 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls\") pod \"maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:53.647354 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:53.647316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:53.785694 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:53.785672 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s"] Jul 16 22:04:53.788160 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:04:53.788127 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2304bd7_28dc_4dbc_9f82_4132cee58617.slice/crio-7638308f02a75e013affe003864ab86bc2a05dd435028390074e5ba51749dce2 WatchSource:0}: Error finding container 7638308f02a75e013affe003864ab86bc2a05dd435028390074e5ba51749dce2: Status 404 returned error can't find the container with id 7638308f02a75e013affe003864ab86bc2a05dd435028390074e5ba51749dce2 Jul 16 22:04:54.774382 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:54.774347 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" event={"ID":"f2304bd7-28dc-4dbc-9f82-4132cee58617","Type":"ContainerStarted","Data":"7638308f02a75e013affe003864ab86bc2a05dd435028390074e5ba51749dce2"} Jul 16 22:04:55.780248 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:55.780218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" event={"ID":"f2304bd7-28dc-4dbc-9f82-4132cee58617","Type":"ContainerStarted","Data":"6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860"} Jul 16 22:04:55.780638 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:55.780292 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:55.799734 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:55.799691 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" podStartSLOduration=1.8829489590000001 podStartE2EDuration="3.799676204s" totalImagesPullingTime="1.916727245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:04:55.796980835 +0000 UTC m=+1561.597867487" watchObservedRunningTime="2026-07-16 22:04:55.799676204 +0000 UTC m=+1561.600562855" Jul 16 22:04:58.125362 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.125328 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s"] Jul 16 22:04:58.125965 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.125546 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" podUID="f2304bd7-28dc-4dbc-9f82-4132cee58617" containerName="maas-api" containerID="cri-o://6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860" gracePeriod=30 Jul 16 22:04:58.375401 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.375348 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:58.443457 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.443432 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppr6k\" (UniqueName: \"kubernetes.io/projected/f2304bd7-28dc-4dbc-9f82-4132cee58617-kube-api-access-ppr6k\") pod \"f2304bd7-28dc-4dbc-9f82-4132cee58617\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " Jul 16 22:04:58.443610 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.443519 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2304bd7-28dc-4dbc-9f82-4132cee58617-openshift-service-ca\") pod \"f2304bd7-28dc-4dbc-9f82-4132cee58617\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " Jul 16 22:04:58.443610 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.443550 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls\") pod \"f2304bd7-28dc-4dbc-9f82-4132cee58617\" (UID: \"f2304bd7-28dc-4dbc-9f82-4132cee58617\") " Jul 16 22:04:58.443895 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.443862 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2304bd7-28dc-4dbc-9f82-4132cee58617-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2304bd7-28dc-4dbc-9f82-4132cee58617" (UID: "f2304bd7-28dc-4dbc-9f82-4132cee58617"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:04:58.445722 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.445697 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2304bd7-28dc-4dbc-9f82-4132cee58617" (UID: "f2304bd7-28dc-4dbc-9f82-4132cee58617"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:04:58.445811 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.445759 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2304bd7-28dc-4dbc-9f82-4132cee58617-kube-api-access-ppr6k" (OuterVolumeSpecName: "kube-api-access-ppr6k") pod "f2304bd7-28dc-4dbc-9f82-4132cee58617" (UID: "f2304bd7-28dc-4dbc-9f82-4132cee58617"). InnerVolumeSpecName "kube-api-access-ppr6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:04:58.544331 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.544294 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2304bd7-28dc-4dbc-9f82-4132cee58617-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:04:58.544331 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.544330 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppr6k\" (UniqueName: \"kubernetes.io/projected/f2304bd7-28dc-4dbc-9f82-4132cee58617-kube-api-access-ppr6k\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:04:58.544331 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.544340 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2304bd7-28dc-4dbc-9f82-4132cee58617-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:04:58.796404 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.796371 2578 generic.go:356] "Generic (PLEG): container finished" podID="f2304bd7-28dc-4dbc-9f82-4132cee58617" containerID="6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860" exitCode=0 Jul 16 22:04:58.796550 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.796416 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" Jul 16 22:04:58.796550 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.796447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" event={"ID":"f2304bd7-28dc-4dbc-9f82-4132cee58617","Type":"ContainerDied","Data":"6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860"} Jul 16 22:04:58.796550 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.796492 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s" event={"ID":"f2304bd7-28dc-4dbc-9f82-4132cee58617","Type":"ContainerDied","Data":"7638308f02a75e013affe003864ab86bc2a05dd435028390074e5ba51749dce2"} Jul 16 22:04:58.796550 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.796514 2578 scope.go:117] "RemoveContainer" containerID="6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860" Jul 16 22:04:58.805829 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.805810 2578 scope.go:117] "RemoveContainer" containerID="6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860" Jul 16 22:04:58.806121 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:04:58.806095 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860\": container with ID starting with 6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860 not found: ID does not exist" containerID="6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860" Jul 16 22:04:58.806174 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.806136 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860"} err="failed to get container status \"6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860\": rpc error: code = NotFound desc = could not find container \"6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860\": container with ID starting with 6adcb85ad8070512548410da3126d3a901bb1f850bedb379c1743a7ef996e860 not found: ID does not exist" Jul 16 22:04:58.821293 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.821272 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s"] Jul 16 22:04:58.827190 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:04:58.827170 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-997f8b52-5cb6ffb8bb-9w72s"] Jul 16 22:05:00.675322 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:00.675289 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2304bd7-28dc-4dbc-9f82-4132cee58617" path="/var/lib/kubelet/pods/f2304bd7-28dc-4dbc-9f82-4132cee58617/volumes" Jul 16 22:05:05.531728 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.531684 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f"] Jul 16 22:05:05.532339 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.532319 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2304bd7-28dc-4dbc-9f82-4132cee58617" containerName="maas-api" Jul 16 22:05:05.532410 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.532341 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2304bd7-28dc-4dbc-9f82-4132cee58617" containerName="maas-api" Jul 16 22:05:05.532457 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.532433 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2304bd7-28dc-4dbc-9f82-4132cee58617" containerName="maas-api" Jul 16 22:05:05.536806 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.536787 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.539335 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.539308 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9bc14df8-serving-cert\"" Jul 16 22:05:05.544774 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.544749 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f"] Jul 16 22:05:05.704996 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.704960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfrkx\" (UniqueName: \"kubernetes.io/projected/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-kube-api-access-hfrkx\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.705157 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.705076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.705157 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.705097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-openshift-service-ca\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.805675 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.805606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.805675 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.805635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-openshift-service-ca\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.805675 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.805668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfrkx\" (UniqueName: \"kubernetes.io/projected/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-kube-api-access-hfrkx\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.805882 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:05:05.805761 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-serving-cert: secret "maas-api-e2e-ait-9bc14df8-serving-cert" not found Jul 16 22:05:05.805882 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:05:05.805848 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls podName:cdf6078d-4c94-40c4-9b77-d3a928b70e5c nodeName:}" failed. No retries permitted until 2026-07-16 22:05:06.305827316 +0000 UTC m=+1572.106713947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls") pod "maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" (UID: "cdf6078d-4c94-40c4-9b77-d3a928b70e5c") : secret "maas-api-e2e-ait-9bc14df8-serving-cert" not found Jul 16 22:05:05.806307 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.806290 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-openshift-service-ca\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:05.817304 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:05.817278 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfrkx\" (UniqueName: \"kubernetes.io/projected/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-kube-api-access-hfrkx\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:06.310689 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:06.310661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:06.313285 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:06.313260 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls\") pod \"maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:06.450713 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:06.450680 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:06.589456 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:06.589415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f"] Jul 16 22:05:06.592085 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:05:06.592052 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf6078d_4c94_40c4_9b77_d3a928b70e5c.slice/crio-0dc147d077f585d65bd43591b868ee8b0bd58e721221634da33b401e4803664d WatchSource:0}: Error finding container 0dc147d077f585d65bd43591b868ee8b0bd58e721221634da33b401e4803664d: Status 404 returned error can't find the container with id 0dc147d077f585d65bd43591b868ee8b0bd58e721221634da33b401e4803664d Jul 16 22:05:06.593404 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:06.593386 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 22:05:06.834038 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:06.834004 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" event={"ID":"cdf6078d-4c94-40c4-9b77-d3a928b70e5c","Type":"ContainerStarted","Data":"0dc147d077f585d65bd43591b868ee8b0bd58e721221634da33b401e4803664d"} Jul 16 22:05:07.841195 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:07.841161 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" event={"ID":"cdf6078d-4c94-40c4-9b77-d3a928b70e5c","Type":"ContainerStarted","Data":"3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b"} Jul 16 22:05:07.841578 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:07.841287 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:07.861248 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:07.861189 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" podStartSLOduration=2.176734398 podStartE2EDuration="2.8611744s" totalImagesPullingTime="684.440002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:05:07.860388032 +0000 UTC m=+1573.661274683" watchObservedRunningTime="2026-07-16 22:05:07.8611744 +0000 UTC m=+1573.662061052" Jul 16 22:05:10.811455 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:10.811428 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f"] Jul 16 22:05:10.811907 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:10.811616 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" podUID="cdf6078d-4c94-40c4-9b77-d3a928b70e5c" containerName="maas-api" containerID="cri-o://3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b" gracePeriod=30 Jul 16 22:05:11.062514 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.062448 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:11.151083 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.151049 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfrkx\" (UniqueName: \"kubernetes.io/projected/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-kube-api-access-hfrkx\") pod \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " Jul 16 22:05:11.151244 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.151099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-openshift-service-ca\") pod \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " Jul 16 22:05:11.151244 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.151125 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls\") pod \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\" (UID: \"cdf6078d-4c94-40c4-9b77-d3a928b70e5c\") " Jul 16 22:05:11.151471 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.151449 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdf6078d-4c94-40c4-9b77-d3a928b70e5c" (UID: "cdf6078d-4c94-40c4-9b77-d3a928b70e5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:05:11.153477 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.153450 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdf6078d-4c94-40c4-9b77-d3a928b70e5c" (UID: "cdf6078d-4c94-40c4-9b77-d3a928b70e5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:05:11.153579 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.153472 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-kube-api-access-hfrkx" (OuterVolumeSpecName: "kube-api-access-hfrkx") pod "cdf6078d-4c94-40c4-9b77-d3a928b70e5c" (UID: "cdf6078d-4c94-40c4-9b77-d3a928b70e5c"). InnerVolumeSpecName "kube-api-access-hfrkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:05:11.252371 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.252347 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfrkx\" (UniqueName: \"kubernetes.io/projected/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-kube-api-access-hfrkx\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:05:11.252371 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.252368 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:05:11.252499 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.252379 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf6078d-4c94-40c4-9b77-d3a928b70e5c-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:05:11.862286 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.862251 2578 generic.go:356] "Generic (PLEG): container finished" podID="cdf6078d-4c94-40c4-9b77-d3a928b70e5c" containerID="3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b" exitCode=0 Jul 16 22:05:11.862706 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.862297 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" Jul 16 22:05:11.862706 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.862341 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" event={"ID":"cdf6078d-4c94-40c4-9b77-d3a928b70e5c","Type":"ContainerDied","Data":"3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b"} Jul 16 22:05:11.862706 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.862390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f" event={"ID":"cdf6078d-4c94-40c4-9b77-d3a928b70e5c","Type":"ContainerDied","Data":"0dc147d077f585d65bd43591b868ee8b0bd58e721221634da33b401e4803664d"} Jul 16 22:05:11.862706 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.862412 2578 scope.go:117] "RemoveContainer" containerID="3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b" Jul 16 22:05:11.871741 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.871725 2578 scope.go:117] "RemoveContainer" containerID="3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b" Jul 16 22:05:11.872014 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:05:11.871993 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b\": container with ID starting with 3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b not found: ID does not exist" containerID="3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b" Jul 16 22:05:11.872062 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.872023 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b"} err="failed to get container status \"3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b\": rpc error: code = NotFound desc = could not find container \"3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b\": container with ID starting with 3c87408b6b25e6be823c0cab925fe89219e0a751c1f34fbb9d2152f2b8d9585b not found: ID does not exist" Jul 16 22:05:11.882652 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.882628 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f"] Jul 16 22:05:11.886859 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:11.886840 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bc14df8-867d9f5c7f-pqs8f"] Jul 16 22:05:12.675011 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:12.674985 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf6078d-4c94-40c4-9b77-d3a928b70e5c" path="/var/lib/kubelet/pods/cdf6078d-4c94-40c4-9b77-d3a928b70e5c/volumes" Jul 16 22:05:23.974444 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.974414 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb"] Jul 16 22:05:23.974953 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.974810 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf6078d-4c94-40c4-9b77-d3a928b70e5c" containerName="maas-api" Jul 16 22:05:23.974953 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.974822 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf6078d-4c94-40c4-9b77-d3a928b70e5c" containerName="maas-api" Jul 16 22:05:23.974953 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.974880 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf6078d-4c94-40c4-9b77-d3a928b70e5c" containerName="maas-api" Jul 16 22:05:23.979100 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.979083 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:23.981375 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.981357 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-257d589c-serving-cert\"" Jul 16 22:05:23.989050 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:23.989029 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb"] Jul 16 22:05:24.052893 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.052865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.053055 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.052938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l8mn\" (UniqueName: \"kubernetes.io/projected/53942cd7-b80c-4a51-8df9-2f15a34d16ff-kube-api-access-7l8mn\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.053055 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.053001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53942cd7-b80c-4a51-8df9-2f15a34d16ff-openshift-service-ca\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.153375 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.153346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53942cd7-b80c-4a51-8df9-2f15a34d16ff-openshift-service-ca\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.153519 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.153399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.153519 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:05:24.153496 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-serving-cert: secret "maas-api-e2e-derive-257d589c-serving-cert" not found Jul 16 22:05:24.153639 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:05:24.153543 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls podName:53942cd7-b80c-4a51-8df9-2f15a34d16ff nodeName:}" failed. No retries permitted until 2026-07-16 22:05:24.653528393 +0000 UTC m=+1590.454415022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls") pod "maas-api-e2e-derive-257d589c-6b4745c594-h72xb" (UID: "53942cd7-b80c-4a51-8df9-2f15a34d16ff") : secret "maas-api-e2e-derive-257d589c-serving-cert" not found Jul 16 22:05:24.153639 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.153609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l8mn\" (UniqueName: \"kubernetes.io/projected/53942cd7-b80c-4a51-8df9-2f15a34d16ff-kube-api-access-7l8mn\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.154041 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.154010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53942cd7-b80c-4a51-8df9-2f15a34d16ff-openshift-service-ca\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.163979 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.163952 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l8mn\" (UniqueName: \"kubernetes.io/projected/53942cd7-b80c-4a51-8df9-2f15a34d16ff-kube-api-access-7l8mn\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.660565 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.660530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.663164 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.663129 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls\") pod \"maas-api-e2e-derive-257d589c-6b4745c594-h72xb\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:24.890030 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:24.889996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:25.049377 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:25.049351 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb"] Jul 16 22:05:25.051753 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:05:25.051714 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53942cd7_b80c_4a51_8df9_2f15a34d16ff.slice/crio-a2fb970a3b46d26d63bc1d145e6faf7f5f8c63062fd4c41599cbb36f2027ede0 WatchSource:0}: Error finding container a2fb970a3b46d26d63bc1d145e6faf7f5f8c63062fd4c41599cbb36f2027ede0: Status 404 returned error can't find the container with id a2fb970a3b46d26d63bc1d145e6faf7f5f8c63062fd4c41599cbb36f2027ede0 Jul 16 22:05:25.934522 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:25.934484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" event={"ID":"53942cd7-b80c-4a51-8df9-2f15a34d16ff","Type":"ContainerStarted","Data":"a2fb970a3b46d26d63bc1d145e6faf7f5f8c63062fd4c41599cbb36f2027ede0"} Jul 16 22:05:26.941044 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:26.941010 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" event={"ID":"53942cd7-b80c-4a51-8df9-2f15a34d16ff","Type":"ContainerStarted","Data":"333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593"} Jul 16 22:05:26.941468 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:26.941158 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:26.961346 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:26.961299 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" podStartSLOduration=2.645244306 podStartE2EDuration="3.961285845s" totalImagesPullingTime="1.316041539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:05:26.96017204 +0000 UTC m=+1592.761058708" watchObservedRunningTime="2026-07-16 22:05:26.961285845 +0000 UTC m=+1592.762172495" Jul 16 22:05:28.522213 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:28.522184 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb"] Jul 16 22:05:28.950104 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:28.950024 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" podUID="53942cd7-b80c-4a51-8df9-2f15a34d16ff" containerName="maas-api" containerID="cri-o://333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593" gracePeriod=30 Jul 16 22:05:29.197925 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.197904 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:29.301958 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.301936 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls\") pod \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " Jul 16 22:05:29.302097 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.301996 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53942cd7-b80c-4a51-8df9-2f15a34d16ff-openshift-service-ca\") pod \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " Jul 16 22:05:29.302097 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.302023 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l8mn\" (UniqueName: \"kubernetes.io/projected/53942cd7-b80c-4a51-8df9-2f15a34d16ff-kube-api-access-7l8mn\") pod \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\" (UID: \"53942cd7-b80c-4a51-8df9-2f15a34d16ff\") " Jul 16 22:05:29.302349 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.302313 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53942cd7-b80c-4a51-8df9-2f15a34d16ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53942cd7-b80c-4a51-8df9-2f15a34d16ff" (UID: "53942cd7-b80c-4a51-8df9-2f15a34d16ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:05:29.304197 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.304171 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53942cd7-b80c-4a51-8df9-2f15a34d16ff" (UID: "53942cd7-b80c-4a51-8df9-2f15a34d16ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:05:29.304197 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.304184 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53942cd7-b80c-4a51-8df9-2f15a34d16ff-kube-api-access-7l8mn" (OuterVolumeSpecName: "kube-api-access-7l8mn") pod "53942cd7-b80c-4a51-8df9-2f15a34d16ff" (UID: "53942cd7-b80c-4a51-8df9-2f15a34d16ff"). InnerVolumeSpecName "kube-api-access-7l8mn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:05:29.403602 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.403548 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53942cd7-b80c-4a51-8df9-2f15a34d16ff-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:05:29.403602 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.403600 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l8mn\" (UniqueName: \"kubernetes.io/projected/53942cd7-b80c-4a51-8df9-2f15a34d16ff-kube-api-access-7l8mn\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:05:29.403749 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.403608 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53942cd7-b80c-4a51-8df9-2f15a34d16ff-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:05:29.955651 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.955617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" event={"ID":"53942cd7-b80c-4a51-8df9-2f15a34d16ff","Type":"ContainerDied","Data":"333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593"} Jul 16 22:05:29.956085 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.955539 2578 generic.go:356] "Generic (PLEG): container finished" podID="53942cd7-b80c-4a51-8df9-2f15a34d16ff" containerID="333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593" exitCode=0 Jul 16 22:05:29.956085 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.956040 2578 scope.go:117] "RemoveContainer" containerID="333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593" Jul 16 22:05:29.956085 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.956064 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" Jul 16 22:05:29.956243 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.956107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb" event={"ID":"53942cd7-b80c-4a51-8df9-2f15a34d16ff","Type":"ContainerDied","Data":"a2fb970a3b46d26d63bc1d145e6faf7f5f8c63062fd4c41599cbb36f2027ede0"} Jul 16 22:05:29.971230 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.971211 2578 scope.go:117] "RemoveContainer" containerID="333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593" Jul 16 22:05:29.971471 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:05:29.971449 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593\": container with ID starting with 333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593 not found: ID does not exist" containerID="333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593" Jul 16 22:05:29.971580 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.971484 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593"} err="failed to get container status \"333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593\": rpc error: code = NotFound desc = could not find container \"333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593\": container with ID starting with 333a9b8421a433fb1c9a1623caf45b242d53da99614930d5d1df31d2c4e57593 not found: ID does not exist" Jul 16 22:05:29.981725 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.981704 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb"] Jul 16 22:05:29.985463 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:29.985442 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-257d589c-6b4745c594-h72xb"] Jul 16 22:05:30.675748 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:05:30.675710 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53942cd7-b80c-4a51-8df9-2f15a34d16ff" path="/var/lib/kubelet/pods/53942cd7-b80c-4a51-8df9-2f15a34d16ff/volumes" Jul 16 22:07:13.005123 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.005091 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz"] Jul 16 22:07:13.005619 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.005421 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53942cd7-b80c-4a51-8df9-2f15a34d16ff" containerName="maas-api" Jul 16 22:07:13.005619 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.005431 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="53942cd7-b80c-4a51-8df9-2f15a34d16ff" containerName="maas-api" Jul 16 22:07:13.005619 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.005501 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="53942cd7-b80c-4a51-8df9-2f15a34d16ff" containerName="maas-api" Jul 16 22:07:13.008453 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.008436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.010838 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.010814 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f76fb58a-serving-cert\"" Jul 16 22:07:13.019580 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.019531 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz"] Jul 16 22:07:13.075771 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.075738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/254eaafd-400b-454a-9cf1-7f80b7b1893d-openshift-service-ca\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.075935 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.075793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/254eaafd-400b-454a-9cf1-7f80b7b1893d-maas-api-tls\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.075935 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.075910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9dqp\" (UniqueName: \"kubernetes.io/projected/254eaafd-400b-454a-9cf1-7f80b7b1893d-kube-api-access-d9dqp\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.176831 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.176796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/254eaafd-400b-454a-9cf1-7f80b7b1893d-openshift-service-ca\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.177019 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.176846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/254eaafd-400b-454a-9cf1-7f80b7b1893d-maas-api-tls\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.177019 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.176888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9dqp\" (UniqueName: \"kubernetes.io/projected/254eaafd-400b-454a-9cf1-7f80b7b1893d-kube-api-access-d9dqp\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.177508 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.177483 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/254eaafd-400b-454a-9cf1-7f80b7b1893d-openshift-service-ca\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.179469 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.179445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/254eaafd-400b-454a-9cf1-7f80b7b1893d-maas-api-tls\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.188002 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.187981 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9dqp\" (UniqueName: \"kubernetes.io/projected/254eaafd-400b-454a-9cf1-7f80b7b1893d-kube-api-access-d9dqp\") pod \"maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.319402 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.319376 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:13.459666 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:13.459644 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz"] Jul 16 22:07:13.462078 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:07:13.462049 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254eaafd_400b_454a_9cf1_7f80b7b1893d.slice/crio-cb46b7a0b94ba8eacd664d1a9ae637459564d63200e66eb79ad4be76ff29056f WatchSource:0}: Error finding container cb46b7a0b94ba8eacd664d1a9ae637459564d63200e66eb79ad4be76ff29056f: Status 404 returned error can't find the container with id cb46b7a0b94ba8eacd664d1a9ae637459564d63200e66eb79ad4be76ff29056f Jul 16 22:07:14.433714 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:14.433685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" event={"ID":"254eaafd-400b-454a-9cf1-7f80b7b1893d","Type":"ContainerStarted","Data":"dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4"} Jul 16 22:07:14.434164 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:14.433732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" event={"ID":"254eaafd-400b-454a-9cf1-7f80b7b1893d","Type":"ContainerStarted","Data":"cb46b7a0b94ba8eacd664d1a9ae637459564d63200e66eb79ad4be76ff29056f"} Jul 16 22:07:14.434164 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:14.433829 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:14.451428 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:14.451375 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" podStartSLOduration=1.953234611 podStartE2EDuration="2.451355572s" totalImagesPullingTime="498.120961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:07:14.450606082 +0000 UTC m=+1700.251492756" watchObservedRunningTime="2026-07-16 22:07:14.451355572 +0000 UTC m=+1700.252242224" Jul 16 22:07:20.443504 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:20.443471 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:25.988979 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:25.988943 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz"] Jul 16 22:07:25.989769 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:25.989697 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" podUID="254eaafd-400b-454a-9cf1-7f80b7b1893d" containerName="maas-api" containerID="cri-o://dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4" gracePeriod=30 Jul 16 22:07:26.233858 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.233837 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:26.296283 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.296249 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/254eaafd-400b-454a-9cf1-7f80b7b1893d-openshift-service-ca\") pod \"254eaafd-400b-454a-9cf1-7f80b7b1893d\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " Jul 16 22:07:26.296283 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.296285 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/254eaafd-400b-454a-9cf1-7f80b7b1893d-maas-api-tls\") pod \"254eaafd-400b-454a-9cf1-7f80b7b1893d\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " Jul 16 22:07:26.296513 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.296329 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9dqp\" (UniqueName: \"kubernetes.io/projected/254eaafd-400b-454a-9cf1-7f80b7b1893d-kube-api-access-d9dqp\") pod \"254eaafd-400b-454a-9cf1-7f80b7b1893d\" (UID: \"254eaafd-400b-454a-9cf1-7f80b7b1893d\") " Jul 16 22:07:26.296645 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.296622 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/254eaafd-400b-454a-9cf1-7f80b7b1893d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "254eaafd-400b-454a-9cf1-7f80b7b1893d" (UID: "254eaafd-400b-454a-9cf1-7f80b7b1893d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:07:26.298613 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.298579 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/254eaafd-400b-454a-9cf1-7f80b7b1893d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "254eaafd-400b-454a-9cf1-7f80b7b1893d" (UID: "254eaafd-400b-454a-9cf1-7f80b7b1893d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:07:26.298613 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.298575 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/254eaafd-400b-454a-9cf1-7f80b7b1893d-kube-api-access-d9dqp" (OuterVolumeSpecName: "kube-api-access-d9dqp") pod "254eaafd-400b-454a-9cf1-7f80b7b1893d" (UID: "254eaafd-400b-454a-9cf1-7f80b7b1893d"). InnerVolumeSpecName "kube-api-access-d9dqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:07:26.397295 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.397264 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/254eaafd-400b-454a-9cf1-7f80b7b1893d-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:07:26.397295 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.397289 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/254eaafd-400b-454a-9cf1-7f80b7b1893d-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:07:26.397295 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.397298 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9dqp\" (UniqueName: \"kubernetes.io/projected/254eaafd-400b-454a-9cf1-7f80b7b1893d-kube-api-access-d9dqp\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:07:26.488148 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.488120 2578 generic.go:356] "Generic (PLEG): container finished" podID="254eaafd-400b-454a-9cf1-7f80b7b1893d" containerID="dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4" exitCode=0 Jul 16 22:07:26.488294 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.488163 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" Jul 16 22:07:26.488294 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.488205 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" event={"ID":"254eaafd-400b-454a-9cf1-7f80b7b1893d","Type":"ContainerDied","Data":"dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4"} Jul 16 22:07:26.488294 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.488257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz" event={"ID":"254eaafd-400b-454a-9cf1-7f80b7b1893d","Type":"ContainerDied","Data":"cb46b7a0b94ba8eacd664d1a9ae637459564d63200e66eb79ad4be76ff29056f"} Jul 16 22:07:26.488294 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.488282 2578 scope.go:117] "RemoveContainer" containerID="dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4" Jul 16 22:07:26.497716 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.497693 2578 scope.go:117] "RemoveContainer" containerID="dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4" Jul 16 22:07:26.497974 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:07:26.497957 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4\": container with ID starting with dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4 not found: ID does not exist" containerID="dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4" Jul 16 22:07:26.498040 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.497986 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4"} err="failed to get container status \"dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4\": rpc error: code = NotFound desc = could not find container \"dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4\": container with ID starting with dc84a2228c4f8a56cb09ed3d6eccd7470dec3f4a2ee336cd7be729fa3b5cf1a4 not found: ID does not exist" Jul 16 22:07:26.510487 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.510458 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz"] Jul 16 22:07:26.519164 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.519145 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f76fb58a-79c4877494-dcbfz"] Jul 16 22:07:26.675431 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:26.675357 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="254eaafd-400b-454a-9cf1-7f80b7b1893d" path="/var/lib/kubelet/pods/254eaafd-400b-454a-9cf1-7f80b7b1893d/volumes" Jul 16 22:07:55.358886 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.358852 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb"] Jul 16 22:07:55.359313 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.359297 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="254eaafd-400b-454a-9cf1-7f80b7b1893d" containerName="maas-api" Jul 16 22:07:55.359357 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.359316 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="254eaafd-400b-454a-9cf1-7f80b7b1893d" containerName="maas-api" Jul 16 22:07:55.359404 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.359395 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="254eaafd-400b-454a-9cf1-7f80b7b1893d" containerName="maas-api" Jul 16 22:07:55.362416 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.362397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.364693 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.364671 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-910fadad-serving-cert\"" Jul 16 22:07:55.372803 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.372775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb"] Jul 16 22:07:55.442736 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.442699 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82bp8\" (UniqueName: \"kubernetes.io/projected/c4049f5c-f734-41d3-8192-9bedc62addcf-kube-api-access-82bp8\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.442736 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.442750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4049f5c-f734-41d3-8192-9bedc62addcf-openshift-service-ca\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.442967 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.442786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4049f5c-f734-41d3-8192-9bedc62addcf-maas-api-tls\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.543333 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.543292 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4049f5c-f734-41d3-8192-9bedc62addcf-maas-api-tls\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.543534 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.543372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82bp8\" (UniqueName: \"kubernetes.io/projected/c4049f5c-f734-41d3-8192-9bedc62addcf-kube-api-access-82bp8\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.543534 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.543398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4049f5c-f734-41d3-8192-9bedc62addcf-openshift-service-ca\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.544084 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.544064 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4049f5c-f734-41d3-8192-9bedc62addcf-openshift-service-ca\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.545925 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.545902 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4049f5c-f734-41d3-8192-9bedc62addcf-maas-api-tls\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.552598 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.552543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82bp8\" (UniqueName: \"kubernetes.io/projected/c4049f5c-f734-41d3-8192-9bedc62addcf-kube-api-access-82bp8\") pod \"maas-api-e2e-mt-910fadad-75fdd7977-8w4rb\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.673253 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.673171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:55.822591 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:55.822449 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb"] Jul 16 22:07:55.825042 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:07:55.825005 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4049f5c_f734_41d3_8192_9bedc62addcf.slice/crio-77ee2a8cc621961840eab99390ef6c799f13d7dc89be2525db5aaf982447bcca WatchSource:0}: Error finding container 77ee2a8cc621961840eab99390ef6c799f13d7dc89be2525db5aaf982447bcca: Status 404 returned error can't find the container with id 77ee2a8cc621961840eab99390ef6c799f13d7dc89be2525db5aaf982447bcca Jul 16 22:07:56.625120 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:56.625023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" event={"ID":"c4049f5c-f734-41d3-8192-9bedc62addcf","Type":"ContainerStarted","Data":"d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602"} Jul 16 22:07:56.625120 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:56.625074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" event={"ID":"c4049f5c-f734-41d3-8192-9bedc62addcf","Type":"ContainerStarted","Data":"77ee2a8cc621961840eab99390ef6c799f13d7dc89be2525db5aaf982447bcca"} Jul 16 22:07:56.625522 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:56.625120 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:07:56.642279 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:07:56.642222 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" podStartSLOduration=1.141072271 podStartE2EDuration="1.642207531s" totalImagesPullingTime="501.13526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:07:56.641313029 +0000 UTC m=+1742.442199682" watchObservedRunningTime="2026-07-16 22:07:56.642207531 +0000 UTC m=+1742.443094181" Jul 16 22:08:02.634898 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:02.634871 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:08:27.110032 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.109997 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb"] Jul 16 22:08:27.110644 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.110263 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" podUID="c4049f5c-f734-41d3-8192-9bedc62addcf" containerName="maas-api" containerID="cri-o://d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602" gracePeriod=30 Jul 16 22:08:27.368948 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.368867 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:08:27.510912 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.510863 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82bp8\" (UniqueName: \"kubernetes.io/projected/c4049f5c-f734-41d3-8192-9bedc62addcf-kube-api-access-82bp8\") pod \"c4049f5c-f734-41d3-8192-9bedc62addcf\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " Jul 16 22:08:27.510912 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.510912 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4049f5c-f734-41d3-8192-9bedc62addcf-openshift-service-ca\") pod \"c4049f5c-f734-41d3-8192-9bedc62addcf\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " Jul 16 22:08:27.511140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.511036 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4049f5c-f734-41d3-8192-9bedc62addcf-maas-api-tls\") pod \"c4049f5c-f734-41d3-8192-9bedc62addcf\" (UID: \"c4049f5c-f734-41d3-8192-9bedc62addcf\") " Jul 16 22:08:27.511362 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.511339 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4049f5c-f734-41d3-8192-9bedc62addcf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4049f5c-f734-41d3-8192-9bedc62addcf" (UID: "c4049f5c-f734-41d3-8192-9bedc62addcf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:08:27.513292 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.513264 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4049f5c-f734-41d3-8192-9bedc62addcf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4049f5c-f734-41d3-8192-9bedc62addcf" (UID: "c4049f5c-f734-41d3-8192-9bedc62addcf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:08:27.513292 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.513280 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4049f5c-f734-41d3-8192-9bedc62addcf-kube-api-access-82bp8" (OuterVolumeSpecName: "kube-api-access-82bp8") pod "c4049f5c-f734-41d3-8192-9bedc62addcf" (UID: "c4049f5c-f734-41d3-8192-9bedc62addcf"). InnerVolumeSpecName "kube-api-access-82bp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:08:27.611986 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.611946 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4049f5c-f734-41d3-8192-9bedc62addcf-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:08:27.611986 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.611979 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82bp8\" (UniqueName: \"kubernetes.io/projected/c4049f5c-f734-41d3-8192-9bedc62addcf-kube-api-access-82bp8\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:08:27.611986 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.611989 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4049f5c-f734-41d3-8192-9bedc62addcf-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:08:27.772034 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.771998 2578 generic.go:356] "Generic (PLEG): container finished" podID="c4049f5c-f734-41d3-8192-9bedc62addcf" containerID="d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602" exitCode=0 Jul 16 22:08:27.772210 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.772051 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" Jul 16 22:08:27.772210 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.772091 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" event={"ID":"c4049f5c-f734-41d3-8192-9bedc62addcf","Type":"ContainerDied","Data":"d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602"} Jul 16 22:08:27.772210 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.772144 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb" event={"ID":"c4049f5c-f734-41d3-8192-9bedc62addcf","Type":"ContainerDied","Data":"77ee2a8cc621961840eab99390ef6c799f13d7dc89be2525db5aaf982447bcca"} Jul 16 22:08:27.772210 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.772177 2578 scope.go:117] "RemoveContainer" containerID="d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602" Jul 16 22:08:27.781452 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.781436 2578 scope.go:117] "RemoveContainer" containerID="d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602" Jul 16 22:08:27.781752 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:08:27.781731 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602\": container with ID starting with d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602 not found: ID does not exist" containerID="d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602" Jul 16 22:08:27.781825 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.781760 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602"} err="failed to get container status \"d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602\": rpc error: code = NotFound desc = could not find container \"d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602\": container with ID starting with d70b06a3a9f313b0ba300ac7a7cd6a95e940e60ca1f68965fd3288790a508602 not found: ID does not exist" Jul 16 22:08:27.794872 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.794838 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb"] Jul 16 22:08:27.800505 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:27.800476 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-910fadad-75fdd7977-8w4rb"] Jul 16 22:08:28.675961 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:28.675928 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4049f5c-f734-41d3-8192-9bedc62addcf" path="/var/lib/kubelet/pods/c4049f5c-f734-41d3-8192-9bedc62addcf/volumes" Jul 16 22:08:54.669426 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:54.669398 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:08:54.674946 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:08:54.674925 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:10:19.082484 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.082395 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb"] Jul 16 22:10:19.083001 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.082823 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4049f5c-f734-41d3-8192-9bedc62addcf" containerName="maas-api" Jul 16 22:10:19.083001 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.082837 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4049f5c-f734-41d3-8192-9bedc62addcf" containerName="maas-api" Jul 16 22:10:19.083001 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.082911 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4049f5c-f734-41d3-8192-9bedc62addcf" containerName="maas-api" Jul 16 22:10:19.086021 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.085998 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.088393 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.088369 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9699fb-serving-cert\"" Jul 16 22:10:19.104289 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.104263 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb"] Jul 16 22:10:19.124523 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.124497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.124646 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.124592 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4dfb\" (UniqueName: \"kubernetes.io/projected/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-kube-api-access-g4dfb\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.124646 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.124618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.225783 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.225744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4dfb\" (UniqueName: \"kubernetes.io/projected/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-kube-api-access-g4dfb\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.226050 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.225791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.226050 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.225827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.226050 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:10:19.225985 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-serving-cert: secret "maas-api-e2e-inf-a-9699fb-serving-cert" not found Jul 16 22:10:19.226242 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:10:19.226074 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls podName:e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb nodeName:}" failed. No retries permitted until 2026-07-16 22:10:19.726051805 +0000 UTC m=+1885.526938434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls") pod "maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" (UID: "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb") : secret "maas-api-e2e-inf-a-9699fb-serving-cert" not found Jul 16 22:10:19.226504 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.226479 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.245136 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.245103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4dfb\" (UniqueName: \"kubernetes.io/projected/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-kube-api-access-g4dfb\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.730428 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.730389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.732970 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.732947 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls\") pod \"maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:19.999071 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:19.999044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:20.141728 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:20.141697 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb"] Jul 16 22:10:20.143849 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:10:20.143819 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79f6fdd_76f3_4fd3_82f5_7adc0f5466cb.slice/crio-76c2f0f0db12ef1202ff2c8ba84853384f6f0ecfeb55d9b93f299de1176b39b9 WatchSource:0}: Error finding container 76c2f0f0db12ef1202ff2c8ba84853384f6f0ecfeb55d9b93f299de1176b39b9: Status 404 returned error can't find the container with id 76c2f0f0db12ef1202ff2c8ba84853384f6f0ecfeb55d9b93f299de1176b39b9 Jul 16 22:10:20.145298 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:20.145280 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 22:10:20.283988 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:20.283951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" event={"ID":"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb","Type":"ContainerStarted","Data":"76c2f0f0db12ef1202ff2c8ba84853384f6f0ecfeb55d9b93f299de1176b39b9"} Jul 16 22:10:21.290655 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:21.290614 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" event={"ID":"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb","Type":"ContainerStarted","Data":"fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604"} Jul 16 22:10:21.291055 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:21.290715 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:21.309594 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:21.309527 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" podStartSLOduration=1.624180443 podStartE2EDuration="2.309512926s" totalImagesPullingTime="685.332483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:10:21.308519155 +0000 UTC m=+1887.109405824" watchObservedRunningTime="2026-07-16 22:10:21.309512926 +0000 UTC m=+1887.110399577" Jul 16 22:10:27.104398 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.104366 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf"] Jul 16 22:10:27.107742 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.107722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.117543 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:10:27.117513 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-inf-b-496a80-serving-cert\" is forbidden: User \"system:node:ip-10-0-136-44.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-44.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-496a80-serving-cert\"" type="*v1.Secret" Jul 16 22:10:27.137363 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.137301 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf"] Jul 16 22:10:27.196968 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.196927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.197165 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.196982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbsp6\" (UniqueName: \"kubernetes.io/projected/dd1e97fd-186b-4d34-8511-892e314330e0-kube-api-access-gbsp6\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.197165 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.197016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd1e97fd-186b-4d34-8511-892e314330e0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.298287 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.298249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.298770 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.298301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbsp6\" (UniqueName: \"kubernetes.io/projected/dd1e97fd-186b-4d34-8511-892e314330e0-kube-api-access-gbsp6\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.298770 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.298335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd1e97fd-186b-4d34-8511-892e314330e0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.299176 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.299153 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd1e97fd-186b-4d34-8511-892e314330e0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:27.302015 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.301997 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:10:27.307803 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:27.307777 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbsp6\" (UniqueName: \"kubernetes.io/projected/dd1e97fd-186b-4d34-8511-892e314330e0-kube-api-access-gbsp6\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:28.299408 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:10:28.299372 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-serving-cert: failed to sync secret cache: timed out waiting for the condition Jul 16 22:10:28.299841 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:10:28.299490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls podName:dd1e97fd-186b-4d34-8511-892e314330e0 nodeName:}" failed. No retries permitted until 2026-07-16 22:10:28.799470521 +0000 UTC m=+1894.600357150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls") pod "maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" (UID: "dd1e97fd-186b-4d34-8511-892e314330e0") : failed to sync secret cache: timed out waiting for the condition Jul 16 22:10:28.617136 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:28.617046 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-496a80-serving-cert\"" Jul 16 22:10:28.812441 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:28.812406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:28.815075 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:28.815047 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:28.919396 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:28.919297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:29.059940 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:29.059915 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf"] Jul 16 22:10:29.061653 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:10:29.061626 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1e97fd_186b_4d34_8511_892e314330e0.slice/crio-214f90381b595ed4935a30c2dd8657066e70770db0459d9fc52f1597e3c0cc59 WatchSource:0}: Error finding container 214f90381b595ed4935a30c2dd8657066e70770db0459d9fc52f1597e3c0cc59: Status 404 returned error can't find the container with id 214f90381b595ed4935a30c2dd8657066e70770db0459d9fc52f1597e3c0cc59 Jul 16 22:10:29.332060 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:29.332019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" event={"ID":"dd1e97fd-186b-4d34-8511-892e314330e0","Type":"ContainerStarted","Data":"214f90381b595ed4935a30c2dd8657066e70770db0459d9fc52f1597e3c0cc59"} Jul 16 22:10:30.340976 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:30.340627 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" event={"ID":"dd1e97fd-186b-4d34-8511-892e314330e0","Type":"ContainerStarted","Data":"dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937"} Jul 16 22:10:30.340976 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:30.340865 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:10:30.362981 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:30.362924 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" podStartSLOduration=3.054341221 podStartE2EDuration="3.362904757s" totalImagesPullingTime="308.563536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:10:30.36283204 +0000 UTC m=+1896.163718712" watchObservedRunningTime="2026-07-16 22:10:30.362904757 +0000 UTC m=+1896.163791409" Jul 16 22:10:36.350439 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:10:36.350410 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:11:47.108384 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.108291 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb"] Jul 16 22:11:47.108958 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.108635 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" podUID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" containerName="maas-api" containerID="cri-o://fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604" gracePeriod=30 Jul 16 22:11:47.358750 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.358682 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:11:47.493032 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.492999 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4dfb\" (UniqueName: \"kubernetes.io/projected/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-kube-api-access-g4dfb\") pod \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " Jul 16 22:11:47.493210 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.493039 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-openshift-service-ca\") pod \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " Jul 16 22:11:47.493210 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.493079 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls\") pod \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\" (UID: \"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb\") " Jul 16 22:11:47.493428 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.493398 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" (UID: "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:11:47.495359 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.495328 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-kube-api-access-g4dfb" (OuterVolumeSpecName: "kube-api-access-g4dfb") pod "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" (UID: "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb"). InnerVolumeSpecName "kube-api-access-g4dfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:11:47.495457 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.495391 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" (UID: "e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:11:47.594044 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.593992 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4dfb\" (UniqueName: \"kubernetes.io/projected/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-kube-api-access-g4dfb\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:11:47.594044 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.594037 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:11:47.594044 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.594048 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:11:47.716764 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.716674 2578 generic.go:356] "Generic (PLEG): container finished" podID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" containerID="fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604" exitCode=0 Jul 16 22:11:47.716764 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.716707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" event={"ID":"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb","Type":"ContainerDied","Data":"fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604"} Jul 16 22:11:47.716764 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.716747 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" Jul 16 22:11:47.717026 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.716786 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" event={"ID":"e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb","Type":"ContainerDied","Data":"76c2f0f0db12ef1202ff2c8ba84853384f6f0ecfeb55d9b93f299de1176b39b9"} Jul 16 22:11:47.717026 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.716816 2578 scope.go:117] "RemoveContainer" containerID="fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604" Jul 16 22:11:47.727125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.727105 2578 scope.go:117] "RemoveContainer" containerID="fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604" Jul 16 22:11:47.727361 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:11:47.727338 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604\": container with ID starting with fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604 not found: ID does not exist" containerID="fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604" Jul 16 22:11:47.727434 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.727375 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604"} err="failed to get container status \"fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604\": rpc error: code = NotFound desc = could not find container \"fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604\": container with ID starting with fe68779afab2df5d671023df50fd9b579dfc2de00c407d5c58e92eb019715604 not found: ID does not exist" Jul 16 22:11:47.741002 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.740970 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb"] Jul 16 22:11:47.747990 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:47.747959 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb"] Jul 16 22:11:48.675733 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:48.675701 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" path="/var/lib/kubelet/pods/e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb/volumes" Jul 16 22:11:50.297264 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:11:50.297227 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9699fb-544586cc4d-b4mvb" podUID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.46:8443/health\": context deadline exceeded" Jul 16 22:12:30.459044 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.459008 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf"] Jul 16 22:12:30.459702 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.459230 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" podUID="dd1e97fd-186b-4d34-8511-892e314330e0" containerName="maas-api" containerID="cri-o://dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937" gracePeriod=30 Jul 16 22:12:30.906924 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.906901 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:12:30.911866 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.911837 2578 generic.go:356] "Generic (PLEG): container finished" podID="dd1e97fd-186b-4d34-8511-892e314330e0" containerID="dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937" exitCode=0 Jul 16 22:12:30.911971 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.911873 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" Jul 16 22:12:30.911971 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.911923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" event={"ID":"dd1e97fd-186b-4d34-8511-892e314330e0","Type":"ContainerDied","Data":"dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937"} Jul 16 22:12:30.912049 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.911979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf" event={"ID":"dd1e97fd-186b-4d34-8511-892e314330e0","Type":"ContainerDied","Data":"214f90381b595ed4935a30c2dd8657066e70770db0459d9fc52f1597e3c0cc59"} Jul 16 22:12:30.912049 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.912000 2578 scope.go:117] "RemoveContainer" containerID="dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937" Jul 16 22:12:30.920242 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.920213 2578 scope.go:117] "RemoveContainer" containerID="dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937" Jul 16 22:12:30.920494 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:12:30.920474 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937\": container with ID starting with dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937 not found: ID does not exist" containerID="dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937" Jul 16 22:12:30.920576 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.920502 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937"} err="failed to get container status \"dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937\": rpc error: code = NotFound desc = could not find container \"dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937\": container with ID starting with dd3916e458f120361eb0e73bbcfdb13704953cd972ddfcb9bb4a2e244326e937 not found: ID does not exist" Jul 16 22:12:30.959888 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.959866 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbsp6\" (UniqueName: \"kubernetes.io/projected/dd1e97fd-186b-4d34-8511-892e314330e0-kube-api-access-gbsp6\") pod \"dd1e97fd-186b-4d34-8511-892e314330e0\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " Jul 16 22:12:30.960002 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.959938 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls\") pod \"dd1e97fd-186b-4d34-8511-892e314330e0\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " Jul 16 22:12:30.960002 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.959957 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd1e97fd-186b-4d34-8511-892e314330e0-openshift-service-ca\") pod \"dd1e97fd-186b-4d34-8511-892e314330e0\" (UID: \"dd1e97fd-186b-4d34-8511-892e314330e0\") " Jul 16 22:12:30.960349 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.960325 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd1e97fd-186b-4d34-8511-892e314330e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dd1e97fd-186b-4d34-8511-892e314330e0" (UID: "dd1e97fd-186b-4d34-8511-892e314330e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:12:30.962006 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.961978 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dd1e97fd-186b-4d34-8511-892e314330e0" (UID: "dd1e97fd-186b-4d34-8511-892e314330e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:12:30.962077 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:30.962054 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd1e97fd-186b-4d34-8511-892e314330e0-kube-api-access-gbsp6" (OuterVolumeSpecName: "kube-api-access-gbsp6") pod "dd1e97fd-186b-4d34-8511-892e314330e0" (UID: "dd1e97fd-186b-4d34-8511-892e314330e0"). InnerVolumeSpecName "kube-api-access-gbsp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:12:31.060841 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:31.060811 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbsp6\" (UniqueName: \"kubernetes.io/projected/dd1e97fd-186b-4d34-8511-892e314330e0-kube-api-access-gbsp6\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:12:31.060841 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:31.060838 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd1e97fd-186b-4d34-8511-892e314330e0-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:12:31.060992 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:31.060849 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd1e97fd-186b-4d34-8511-892e314330e0-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:12:31.233680 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:31.233652 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf"] Jul 16 22:12:31.237604 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:31.237577 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-496a80-5cb5d546dd-8w9rf"] Jul 16 22:12:32.675385 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:32.675342 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd1e97fd-186b-4d34-8511-892e314330e0" path="/var/lib/kubelet/pods/dd1e97fd-186b-4d34-8511-892e314330e0/volumes" Jul 16 22:12:39.850652 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.850616 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656"] Jul 16 22:12:39.851125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.850959 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" containerName="maas-api" Jul 16 22:12:39.851125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.850970 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" containerName="maas-api" Jul 16 22:12:39.851125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.850988 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1e97fd-186b-4d34-8511-892e314330e0" containerName="maas-api" Jul 16 22:12:39.851125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.850995 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1e97fd-186b-4d34-8511-892e314330e0" containerName="maas-api" Jul 16 22:12:39.851125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.851046 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e79f6fdd-76f3-4fd3-82f5-7adc0f5466cb" containerName="maas-api" Jul 16 22:12:39.851125 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.851055 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd1e97fd-186b-4d34-8511-892e314330e0" containerName="maas-api" Jul 16 22:12:39.855307 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.855289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:39.858405 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.858361 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fc2964-serving-cert\"" Jul 16 22:12:39.866775 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.866750 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656"] Jul 16 22:12:39.939532 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.939495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-maas-api-tls\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:39.939532 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.939535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjknl\" (UniqueName: \"kubernetes.io/projected/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-kube-api-access-tjknl\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:39.939773 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:39.939637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-openshift-service-ca\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.040685 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.040639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-openshift-service-ca\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.040901 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.040768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-maas-api-tls\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.040901 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.040801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjknl\" (UniqueName: \"kubernetes.io/projected/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-kube-api-access-tjknl\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.041460 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.041434 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-openshift-service-ca\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.043482 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.043451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-maas-api-tls\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.050717 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.050694 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjknl\" (UniqueName: \"kubernetes.io/projected/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-kube-api-access-tjknl\") pod \"maas-api-e2e-api-a-fc2964-86bbf76456-7m656\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.167623 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.167501 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:40.307538 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.307517 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656"] Jul 16 22:12:40.309918 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:12:40.309892 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6edc8fa9_836f_4ccf_8ab9_ed7022c91d73.slice/crio-34619e017ebb08070facbded5d3b7b6852b0dec02a45cd378626d2817c02fabc WatchSource:0}: Error finding container 34619e017ebb08070facbded5d3b7b6852b0dec02a45cd378626d2817c02fabc: Status 404 returned error can't find the container with id 34619e017ebb08070facbded5d3b7b6852b0dec02a45cd378626d2817c02fabc Jul 16 22:12:40.965214 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:40.965174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" event={"ID":"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73","Type":"ContainerStarted","Data":"34619e017ebb08070facbded5d3b7b6852b0dec02a45cd378626d2817c02fabc"} Jul 16 22:12:41.971797 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:41.971761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" event={"ID":"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73","Type":"ContainerStarted","Data":"fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b"} Jul 16 22:12:41.972245 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:41.971863 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:41.992613 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:41.992547 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" podStartSLOduration=2.3331986860000002 podStartE2EDuration="2.992532406s" totalImagesPullingTime="659.33372ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:12:41.991090639 +0000 UTC m=+2027.791977314" watchObservedRunningTime="2026-07-16 22:12:41.992532406 +0000 UTC m=+2027.793419121" Jul 16 22:12:47.801100 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.801063 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp"] Jul 16 22:12:47.805228 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.805207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:47.807825 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.807803 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f82999-serving-cert\"" Jul 16 22:12:47.823277 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.823257 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp"] Jul 16 22:12:47.905786 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.905750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f01b62-f25d-417d-98cb-088ddf1e5f62-openshift-service-ca\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:47.906034 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.905797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq75x\" (UniqueName: \"kubernetes.io/projected/00f01b62-f25d-417d-98cb-088ddf1e5f62-kube-api-access-sq75x\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:47.906034 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.905972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:47.982082 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:47.982050 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:12:48.007205 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:48.007153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:48.007380 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:48.007225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f01b62-f25d-417d-98cb-088ddf1e5f62-openshift-service-ca\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:48.007380 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:48.007271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq75x\" (UniqueName: \"kubernetes.io/projected/00f01b62-f25d-417d-98cb-088ddf1e5f62-kube-api-access-sq75x\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:48.007380 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:12:48.007294 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-serving-cert: secret "maas-api-e2e-api-b-f82999-serving-cert" not found Jul 16 22:12:48.007380 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:12:48.007346 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls podName:00f01b62-f25d-417d-98cb-088ddf1e5f62 nodeName:}" failed. No retries permitted until 2026-07-16 22:12:48.507331967 +0000 UTC m=+2034.308218596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls") pod "maas-api-e2e-api-b-f82999-79f66646b-7hbkp" (UID: "00f01b62-f25d-417d-98cb-088ddf1e5f62") : secret "maas-api-e2e-api-b-f82999-serving-cert" not found Jul 16 22:12:48.008036 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:48.008013 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f01b62-f25d-417d-98cb-088ddf1e5f62-openshift-service-ca\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:48.025199 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:48.025170 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq75x\" (UniqueName: \"kubernetes.io/projected/00f01b62-f25d-417d-98cb-088ddf1e5f62-kube-api-access-sq75x\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:48.512168 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:48.512125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:48.512348 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:12:48.512267 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-serving-cert: secret "maas-api-e2e-api-b-f82999-serving-cert" not found Jul 16 22:12:48.512348 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:12:48.512331 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls podName:00f01b62-f25d-417d-98cb-088ddf1e5f62 nodeName:}" failed. No retries permitted until 2026-07-16 22:12:49.512315751 +0000 UTC m=+2035.313202380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls") pod "maas-api-e2e-api-b-f82999-79f66646b-7hbkp" (UID: "00f01b62-f25d-417d-98cb-088ddf1e5f62") : secret "maas-api-e2e-api-b-f82999-serving-cert" not found Jul 16 22:12:49.521309 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:49.521278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:49.523874 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:49.523854 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") pod \"maas-api-e2e-api-b-f82999-79f66646b-7hbkp\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:49.615763 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:49.615728 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:49.755516 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:49.755487 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp"] Jul 16 22:12:49.757884 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:12:49.757858 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f01b62_f25d_417d_98cb_088ddf1e5f62.slice/crio-bc8925bf94c5879b61415105e23e2202d629cb4eef0ad61c39cbe129241c5c01 WatchSource:0}: Error finding container bc8925bf94c5879b61415105e23e2202d629cb4eef0ad61c39cbe129241c5c01: Status 404 returned error can't find the container with id bc8925bf94c5879b61415105e23e2202d629cb4eef0ad61c39cbe129241c5c01 Jul 16 22:12:50.010249 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:50.010222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" event={"ID":"00f01b62-f25d-417d-98cb-088ddf1e5f62","Type":"ContainerStarted","Data":"bc8925bf94c5879b61415105e23e2202d629cb4eef0ad61c39cbe129241c5c01"} Jul 16 22:12:51.015920 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:51.015889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" event={"ID":"00f01b62-f25d-417d-98cb-088ddf1e5f62","Type":"ContainerStarted","Data":"1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84"} Jul 16 22:12:51.016325 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:51.015949 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:12:51.036013 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:51.035967 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" podStartSLOduration=3.701530442 podStartE2EDuration="4.035953252s" totalImagesPullingTime="334.42281ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:12:51.033444669 +0000 UTC m=+2036.834331331" watchObservedRunningTime="2026-07-16 22:12:51.035953252 +0000 UTC m=+2036.836839903" Jul 16 22:12:57.026242 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:12:57.026217 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:13:01.005366 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.005333 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656"] Jul 16 22:13:01.005918 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.005601 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" podUID="6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" containerName="maas-api" containerID="cri-o://fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b" gracePeriod=30 Jul 16 22:13:01.278062 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.278037 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:13:01.326013 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.325987 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-openshift-service-ca\") pod \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " Jul 16 22:13:01.326157 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.326046 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjknl\" (UniqueName: \"kubernetes.io/projected/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-kube-api-access-tjknl\") pod \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " Jul 16 22:13:01.326157 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.326066 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-maas-api-tls\") pod \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\" (UID: \"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73\") " Jul 16 22:13:01.326285 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.326173 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" (UID: "6edc8fa9-836f-4ccf-8ab9-ed7022c91d73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:13:01.326343 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.326286 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:13:01.328203 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.328180 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-kube-api-access-tjknl" (OuterVolumeSpecName: "kube-api-access-tjknl") pod "6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" (UID: "6edc8fa9-836f-4ccf-8ab9-ed7022c91d73"). InnerVolumeSpecName "kube-api-access-tjknl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:13:01.328331 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.328290 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" (UID: "6edc8fa9-836f-4ccf-8ab9-ed7022c91d73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:13:01.427310 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.427285 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjknl\" (UniqueName: \"kubernetes.io/projected/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-kube-api-access-tjknl\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:13:01.427310 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:01.427308 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:13:02.066836 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.066802 2578 generic.go:356] "Generic (PLEG): container finished" podID="6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" containerID="fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b" exitCode=0 Jul 16 22:13:02.067226 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.066851 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" Jul 16 22:13:02.067226 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.066849 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" event={"ID":"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73","Type":"ContainerDied","Data":"fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b"} Jul 16 22:13:02.067226 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.066983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656" event={"ID":"6edc8fa9-836f-4ccf-8ab9-ed7022c91d73","Type":"ContainerDied","Data":"34619e017ebb08070facbded5d3b7b6852b0dec02a45cd378626d2817c02fabc"} Jul 16 22:13:02.067226 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.067017 2578 scope.go:117] "RemoveContainer" containerID="fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b" Jul 16 22:13:02.076600 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.076545 2578 scope.go:117] "RemoveContainer" containerID="fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b" Jul 16 22:13:02.076869 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:13:02.076851 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b\": container with ID starting with fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b not found: ID does not exist" containerID="fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b" Jul 16 22:13:02.076922 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.076879 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b"} err="failed to get container status \"fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b\": rpc error: code = NotFound desc = could not find container \"fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b\": container with ID starting with fffc13dac885700867a0ed32d10923c537ae6ad1c6708c2de37300e2a043e37b not found: ID does not exist" Jul 16 22:13:02.093401 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.093378 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656"] Jul 16 22:13:02.101788 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.101769 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fc2964-86bbf76456-7m656"] Jul 16 22:13:02.676049 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:02.676016 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" path="/var/lib/kubelet/pods/6edc8fa9-836f-4ccf-8ab9-ed7022c91d73/volumes" Jul 16 22:13:08.086652 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.086612 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp"] Jul 16 22:13:08.087095 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.086821 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" podUID="00f01b62-f25d-417d-98cb-088ddf1e5f62" containerName="maas-api" containerID="cri-o://1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84" gracePeriod=30 Jul 16 22:13:08.338835 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.338773 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:13:08.380159 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.380129 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") pod \"00f01b62-f25d-417d-98cb-088ddf1e5f62\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " Jul 16 22:13:08.380353 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.380168 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq75x\" (UniqueName: \"kubernetes.io/projected/00f01b62-f25d-417d-98cb-088ddf1e5f62-kube-api-access-sq75x\") pod \"00f01b62-f25d-417d-98cb-088ddf1e5f62\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " Jul 16 22:13:08.380353 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.380198 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f01b62-f25d-417d-98cb-088ddf1e5f62-openshift-service-ca\") pod \"00f01b62-f25d-417d-98cb-088ddf1e5f62\" (UID: \"00f01b62-f25d-417d-98cb-088ddf1e5f62\") " Jul 16 22:13:08.380633 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.380598 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f01b62-f25d-417d-98cb-088ddf1e5f62-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00f01b62-f25d-417d-98cb-088ddf1e5f62" (UID: "00f01b62-f25d-417d-98cb-088ddf1e5f62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:13:08.382407 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.382372 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f01b62-f25d-417d-98cb-088ddf1e5f62-kube-api-access-sq75x" (OuterVolumeSpecName: "kube-api-access-sq75x") pod "00f01b62-f25d-417d-98cb-088ddf1e5f62" (UID: "00f01b62-f25d-417d-98cb-088ddf1e5f62"). InnerVolumeSpecName "kube-api-access-sq75x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:13:08.382847 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.382823 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00f01b62-f25d-417d-98cb-088ddf1e5f62" (UID: "00f01b62-f25d-417d-98cb-088ddf1e5f62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:13:08.480862 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.480828 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00f01b62-f25d-417d-98cb-088ddf1e5f62-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:13:08.480862 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.480856 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq75x\" (UniqueName: \"kubernetes.io/projected/00f01b62-f25d-417d-98cb-088ddf1e5f62-kube-api-access-sq75x\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:13:08.480862 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:08.480866 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00f01b62-f25d-417d-98cb-088ddf1e5f62-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:13:09.101248 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.101207 2578 generic.go:356] "Generic (PLEG): container finished" podID="00f01b62-f25d-417d-98cb-088ddf1e5f62" containerID="1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84" exitCode=0 Jul 16 22:13:09.101248 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.101249 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" Jul 16 22:13:09.101814 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.101300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" event={"ID":"00f01b62-f25d-417d-98cb-088ddf1e5f62","Type":"ContainerDied","Data":"1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84"} Jul 16 22:13:09.101814 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.101352 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp" event={"ID":"00f01b62-f25d-417d-98cb-088ddf1e5f62","Type":"ContainerDied","Data":"bc8925bf94c5879b61415105e23e2202d629cb4eef0ad61c39cbe129241c5c01"} Jul 16 22:13:09.101814 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.101375 2578 scope.go:117] "RemoveContainer" containerID="1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84" Jul 16 22:13:09.109823 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.109807 2578 scope.go:117] "RemoveContainer" containerID="1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84" Jul 16 22:13:09.110074 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:13:09.110058 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84\": container with ID starting with 1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84 not found: ID does not exist" containerID="1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84" Jul 16 22:13:09.110127 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.110083 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84"} err="failed to get container status \"1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84\": rpc error: code = NotFound desc = could not find container \"1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84\": container with ID starting with 1c5ea2c6d99d901c671bbe1a6050e7a23d15b67119fef531339957272027de84 not found: ID does not exist" Jul 16 22:13:09.117924 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.117890 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp"] Jul 16 22:13:09.122497 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:09.122467 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f82999-79f66646b-7hbkp"] Jul 16 22:13:10.676096 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:10.676054 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f01b62-f25d-417d-98cb-088ddf1e5f62" path="/var/lib/kubelet/pods/00f01b62-f25d-417d-98cb-088ddf1e5f62/volumes" Jul 16 22:13:17.983449 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983374 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4"] Jul 16 22:13:17.983849 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983754 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" containerName="maas-api" Jul 16 22:13:17.983849 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983766 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" containerName="maas-api" Jul 16 22:13:17.983849 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983778 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f01b62-f25d-417d-98cb-088ddf1e5f62" containerName="maas-api" Jul 16 22:13:17.983849 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983783 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f01b62-f25d-417d-98cb-088ddf1e5f62" containerName="maas-api" Jul 16 22:13:17.984009 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983863 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6edc8fa9-836f-4ccf-8ab9-ed7022c91d73" containerName="maas-api" Jul 16 22:13:17.984009 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.983875 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f01b62-f25d-417d-98cb-088ddf1e5f62" containerName="maas-api" Jul 16 22:13:17.988215 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.988196 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:17.990793 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.990769 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5d61a3-serving-cert\"" Jul 16 22:13:17.998702 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:17.998677 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4"] Jul 16 22:13:18.060261 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.060232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.060412 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.060270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kltqm\" (UniqueName: \"kubernetes.io/projected/8dcb7973-6dd9-442b-97da-5ce6b0b56481-kube-api-access-kltqm\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.060412 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.060296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcb7973-6dd9-442b-97da-5ce6b0b56481-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.161135 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.161101 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.161135 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.161139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kltqm\" (UniqueName: \"kubernetes.io/projected/8dcb7973-6dd9-442b-97da-5ce6b0b56481-kube-api-access-kltqm\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.161353 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.161174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcb7973-6dd9-442b-97da-5ce6b0b56481-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.161353 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:13:18.161256 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-serving-cert: secret "maas-api-e2e-shared-a-5d61a3-serving-cert" not found Jul 16 22:13:18.161353 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:13:18.161325 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls podName:8dcb7973-6dd9-442b-97da-5ce6b0b56481 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:18.661303191 +0000 UTC m=+2064.462189820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls") pod "maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" (UID: "8dcb7973-6dd9-442b-97da-5ce6b0b56481") : secret "maas-api-e2e-shared-a-5d61a3-serving-cert" not found Jul 16 22:13:18.162100 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.162073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcb7973-6dd9-442b-97da-5ce6b0b56481-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.170002 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.169975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kltqm\" (UniqueName: \"kubernetes.io/projected/8dcb7973-6dd9-442b-97da-5ce6b0b56481-kube-api-access-kltqm\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.665864 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.665837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.668490 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.668456 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls\") pod \"maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:18.899111 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:18.899066 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:19.041705 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:19.041676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4"] Jul 16 22:13:19.044344 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:13:19.044316 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dcb7973_6dd9_442b_97da_5ce6b0b56481.slice/crio-fb1df0586f8eb078ca38a1b7b57ecada5e97c7ac2b5312eff40da8221e160468 WatchSource:0}: Error finding container fb1df0586f8eb078ca38a1b7b57ecada5e97c7ac2b5312eff40da8221e160468: Status 404 returned error can't find the container with id fb1df0586f8eb078ca38a1b7b57ecada5e97c7ac2b5312eff40da8221e160468 Jul 16 22:13:19.148618 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:19.148586 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" event={"ID":"8dcb7973-6dd9-442b-97da-5ce6b0b56481","Type":"ContainerStarted","Data":"fb1df0586f8eb078ca38a1b7b57ecada5e97c7ac2b5312eff40da8221e160468"} Jul 16 22:13:20.156247 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:20.156213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" event={"ID":"8dcb7973-6dd9-442b-97da-5ce6b0b56481","Type":"ContainerStarted","Data":"16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34"} Jul 16 22:13:20.156706 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:20.156275 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:20.174650 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:20.174598 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" podStartSLOduration=2.857613226 podStartE2EDuration="3.17458038s" totalImagesPullingTime="316.967154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:13:20.173091412 +0000 UTC m=+2065.973978087" watchObservedRunningTime="2026-07-16 22:13:20.17458038 +0000 UTC m=+2065.975467028" Jul 16 22:13:26.025988 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.025955 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4"] Jul 16 22:13:26.029840 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.029819 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.032455 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.032433 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ff5f80-serving-cert\"" Jul 16 22:13:26.043423 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.043402 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4"] Jul 16 22:13:26.132402 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.132373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.132402 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.132405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6be80e4b-9a72-48e4-aee5-058aa6c54598-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.132614 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.132444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fqtt\" (UniqueName: \"kubernetes.io/projected/6be80e4b-9a72-48e4-aee5-058aa6c54598-kube-api-access-5fqtt\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.166102 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.166082 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:13:26.233059 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.233016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.233059 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.233052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6be80e4b-9a72-48e4-aee5-058aa6c54598-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.233290 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.233097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fqtt\" (UniqueName: \"kubernetes.io/projected/6be80e4b-9a72-48e4-aee5-058aa6c54598-kube-api-access-5fqtt\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.233290 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:13:26.233161 2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-serving-cert: secret "maas-api-e2e-shared-b-ff5f80-serving-cert" not found Jul 16 22:13:26.233290 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:13:26.233240 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls podName:6be80e4b-9a72-48e4-aee5-058aa6c54598 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:26.733216937 +0000 UTC m=+2072.534103585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls") pod "maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" (UID: "6be80e4b-9a72-48e4-aee5-058aa6c54598") : secret "maas-api-e2e-shared-b-ff5f80-serving-cert" not found Jul 16 22:13:26.233940 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.233913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6be80e4b-9a72-48e4-aee5-058aa6c54598-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.243726 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.243703 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fqtt\" (UniqueName: \"kubernetes.io/projected/6be80e4b-9a72-48e4-aee5-058aa6c54598-kube-api-access-5fqtt\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.739355 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.739322 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.741970 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.741939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls\") pod \"maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:26.940949 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:26.940918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:27.078786 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:27.078764 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4"] Jul 16 22:13:27.080981 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:13:27.080949 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be80e4b_9a72_48e4_aee5_058aa6c54598.slice/crio-cf9c4e1199f4a74741cf6fdfc36c06b9885e7e9997219539cfd1123414b68a5b WatchSource:0}: Error finding container cf9c4e1199f4a74741cf6fdfc36c06b9885e7e9997219539cfd1123414b68a5b: Status 404 returned error can't find the container with id cf9c4e1199f4a74741cf6fdfc36c06b9885e7e9997219539cfd1123414b68a5b Jul 16 22:13:27.190465 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:27.190429 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" event={"ID":"6be80e4b-9a72-48e4-aee5-058aa6c54598","Type":"ContainerStarted","Data":"cf9c4e1199f4a74741cf6fdfc36c06b9885e7e9997219539cfd1123414b68a5b"} Jul 16 22:13:28.199515 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:28.199473 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" event={"ID":"6be80e4b-9a72-48e4-aee5-058aa6c54598","Type":"ContainerStarted","Data":"a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e"} Jul 16 22:13:28.199982 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:28.199616 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:28.218401 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:28.218353 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" podStartSLOduration=1.850333189 podStartE2EDuration="2.218339942s" totalImagesPullingTime="368.006753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:13:28.216697673 +0000 UTC m=+2074.017584329" watchObservedRunningTime="2026-07-16 22:13:28.218339942 +0000 UTC m=+2074.019226593" Jul 16 22:13:34.209911 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:34.209879 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:13:54.697307 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:54.697283 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:13:54.703713 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:13:54.703692 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:15:00.140748 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.140665 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw"] Jul 16 22:15:00.144409 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.144386 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9"] Jul 16 22:15:00.144540 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.144501 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" Jul 16 22:15:00.147174 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.147152 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7tspb\"" Jul 16 22:15:00.147674 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.147657 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn"] Jul 16 22:15:00.147792 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.147777 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" Jul 16 22:15:00.150971 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.150952 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" Jul 16 22:15:00.153358 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.153335 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw"] Jul 16 22:15:00.157774 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.157755 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" Jul 16 22:15:00.158902 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.158887 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9"] Jul 16 22:15:00.161694 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.161046 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn"] Jul 16 22:15:00.168908 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.168892 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" Jul 16 22:15:00.204222 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.204190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" Jul 16 22:15:00.331836 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.331796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw"] Jul 16 22:15:00.335352 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:15:00.335135 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba3c995b_32f6_4934_a9d1_a0bc562cbf90.slice/crio-42b81a807257a1b9adf51b12655302fe34a98f8ed6f9d93d703d031692f5d337 WatchSource:0}: Error finding container 42b81a807257a1b9adf51b12655302fe34a98f8ed6f9d93d703d031692f5d337: Status 404 returned error can't find the container with id 42b81a807257a1b9adf51b12655302fe34a98f8ed6f9d93d703d031692f5d337 Jul 16 22:15:00.359769 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.359741 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9"] Jul 16 22:15:00.362941 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:15:00.362914 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0450ce4_c13c_4e66_af92_b3011167f7bc.slice/crio-01063e95b14b07a867fcafa2f853a09b6d87d6826782e81bc712925673f328d3 WatchSource:0}: Error finding container 01063e95b14b07a867fcafa2f853a09b6d87d6826782e81bc712925673f328d3: Status 404 returned error can't find the container with id 01063e95b14b07a867fcafa2f853a09b6d87d6826782e81bc712925673f328d3 Jul 16 22:15:00.389900 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.389879 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn"] Jul 16 22:15:00.396233 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:15:00.396209 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e0aa37_d000_476c_bf5f_622d7f9ab616.slice/crio-953d04564ec980ddd64e67722d7882255c1ef84451fc6fd9b5c5e05fa9652edf WatchSource:0}: Error finding container 953d04564ec980ddd64e67722d7882255c1ef84451fc6fd9b5c5e05fa9652edf: Status 404 returned error can't find the container with id 953d04564ec980ddd64e67722d7882255c1ef84451fc6fd9b5c5e05fa9652edf Jul 16 22:15:00.637038 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.636999 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" event={"ID":"ba3c995b-32f6-4934-a9d1-a0bc562cbf90","Type":"ContainerStarted","Data":"f79928e862a43dc976ceb999791cd6459a515b59120282e575c03d584127f50a"} Jul 16 22:15:00.637234 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.637071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" event={"ID":"ba3c995b-32f6-4934-a9d1-a0bc562cbf90","Type":"ContainerStarted","Data":"42b81a807257a1b9adf51b12655302fe34a98f8ed6f9d93d703d031692f5d337"} Jul 16 22:15:00.638327 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.638305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" event={"ID":"a0450ce4-c13c-4e66-af92-b3011167f7bc","Type":"ContainerStarted","Data":"01063e95b14b07a867fcafa2f853a09b6d87d6826782e81bc712925673f328d3"} Jul 16 22:15:00.639473 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.639451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" event={"ID":"d9e0aa37-d000-476c-bf5f-622d7f9ab616","Type":"ContainerStarted","Data":"953d04564ec980ddd64e67722d7882255c1ef84451fc6fd9b5c5e05fa9652edf"} Jul 16 22:15:00.655288 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:00.655198 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" podStartSLOduration=0.655184336 podStartE2EDuration="655.184336ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:00.653795515 +0000 UTC m=+2166.454682169" watchObservedRunningTime="2026-07-16 22:15:00.655184336 +0000 UTC m=+2166.456070988" Jul 16 22:15:01.646150 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:01.646108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" event={"ID":"a0450ce4-c13c-4e66-af92-b3011167f7bc","Type":"ContainerStarted","Data":"fd29ed13a121f8869082fbf030c9d44da1ea8fa35ac3a43904756110a95bd02c"} Jul 16 22:15:01.647516 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:01.647489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" event={"ID":"d9e0aa37-d000-476c-bf5f-622d7f9ab616","Type":"ContainerStarted","Data":"69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09"} Jul 16 22:15:01.665113 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:01.665066 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" podStartSLOduration=1.665051814 podStartE2EDuration="1.665051814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:01.663590478 +0000 UTC m=+2167.464477152" watchObservedRunningTime="2026-07-16 22:15:01.665051814 +0000 UTC m=+2167.465938528" Jul 16 22:15:01.679040 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:15:01.678996 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" podStartSLOduration=1.678982726 podStartE2EDuration="1.678982726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:01.677618474 +0000 UTC m=+2167.478505142" watchObservedRunningTime="2026-07-16 22:15:01.678982726 +0000 UTC m=+2167.479869376" Jul 16 22:16:00.011128 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:00.011099 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9"] Jul 16 22:16:00.011840 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:00.011387 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" podUID="a0450ce4-c13c-4e66-af92-b3011167f7bc" containerName="cleanup" containerID="cri-o://fd29ed13a121f8869082fbf030c9d44da1ea8fa35ac3a43904756110a95bd02c" gracePeriod=30 Jul 16 22:16:00.014412 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:00.014381 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw"] Jul 16 22:16:00.014612 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:00.014550 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" podUID="ba3c995b-32f6-4934-a9d1-a0bc562cbf90" containerName="cleanup" containerID="cri-o://f79928e862a43dc976ceb999791cd6459a515b59120282e575c03d584127f50a" gracePeriod=30 Jul 16 22:16:00.016447 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:00.016417 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn"] Jul 16 22:16:00.016610 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:00.016593 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" podUID="d9e0aa37-d000-476c-bf5f-622d7f9ab616" containerName="cleanup" containerID="cri-o://69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09" gracePeriod=30 Jul 16 22:16:30.067937 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.067900 2578 generic.go:356] "Generic (PLEG): container finished" podID="ba3c995b-32f6-4934-a9d1-a0bc562cbf90" containerID="f79928e862a43dc976ceb999791cd6459a515b59120282e575c03d584127f50a" exitCode=137 Jul 16 22:16:30.067937 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.067922 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" event={"ID":"ba3c995b-32f6-4934-a9d1-a0bc562cbf90","Type":"ContainerDied","Data":"f79928e862a43dc976ceb999791cd6459a515b59120282e575c03d584127f50a"} Jul 16 22:16:30.069810 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.069777 2578 generic.go:356] "Generic (PLEG): container finished" podID="a0450ce4-c13c-4e66-af92-b3011167f7bc" containerID="fd29ed13a121f8869082fbf030c9d44da1ea8fa35ac3a43904756110a95bd02c" exitCode=137 Jul 16 22:16:30.069946 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.069811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" event={"ID":"a0450ce4-c13c-4e66-af92-b3011167f7bc","Type":"ContainerDied","Data":"fd29ed13a121f8869082fbf030c9d44da1ea8fa35ac3a43904756110a95bd02c"} Jul 16 22:16:30.227487 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.227462 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" Jul 16 22:16:30.246993 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.246968 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" Jul 16 22:16:30.250664 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:30.250643 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" Jul 16 22:16:31.077114 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.077077 2578 generic.go:356] "Generic (PLEG): container finished" podID="d9e0aa37-d000-476c-bf5f-622d7f9ab616" containerID="69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09" exitCode=137 Jul 16 22:16:31.077505 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.077136 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" Jul 16 22:16:31.077505 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.077165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" event={"ID":"d9e0aa37-d000-476c-bf5f-622d7f9ab616","Type":"ContainerDied","Data":"69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09"} Jul 16 22:16:31.077505 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.077252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn" event={"ID":"d9e0aa37-d000-476c-bf5f-622d7f9ab616","Type":"ContainerDied","Data":"953d04564ec980ddd64e67722d7882255c1ef84451fc6fd9b5c5e05fa9652edf"} Jul 16 22:16:31.077505 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.077286 2578 scope.go:117] "RemoveContainer" containerID="69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09" Jul 16 22:16:31.078783 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.078515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" event={"ID":"ba3c995b-32f6-4934-a9d1-a0bc562cbf90","Type":"ContainerDied","Data":"42b81a807257a1b9adf51b12655302fe34a98f8ed6f9d93d703d031692f5d337"} Jul 16 22:16:31.078783 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.078524 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw" Jul 16 22:16:31.080409 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.080390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" event={"ID":"a0450ce4-c13c-4e66-af92-b3011167f7bc","Type":"ContainerDied","Data":"01063e95b14b07a867fcafa2f853a09b6d87d6826782e81bc712925673f328d3"} Jul 16 22:16:31.080534 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.080491 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9" Jul 16 22:16:31.087198 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.087184 2578 scope.go:117] "RemoveContainer" containerID="69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09" Jul 16 22:16:31.087454 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:16:31.087434 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09\": container with ID starting with 69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09 not found: ID does not exist" containerID="69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09" Jul 16 22:16:31.087525 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.087465 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09"} err="failed to get container status \"69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09\": rpc error: code = NotFound desc = could not find container \"69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09\": container with ID starting with 69b53c526894f4bb6939dbe95ba2b054ce1a38cb1df59cdf7e86c67161a91e09 not found: ID does not exist" Jul 16 22:16:31.087525 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.087488 2578 scope.go:117] "RemoveContainer" containerID="f79928e862a43dc976ceb999791cd6459a515b59120282e575c03d584127f50a" Jul 16 22:16:31.094513 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.094490 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn"] Jul 16 22:16:31.095450 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.095435 2578 scope.go:117] "RemoveContainer" containerID="fd29ed13a121f8869082fbf030c9d44da1ea8fa35ac3a43904756110a95bd02c" Jul 16 22:16:31.097885 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.097864 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ff5f80-29737335-4hsdn"] Jul 16 22:16:31.107449 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.107429 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9"] Jul 16 22:16:31.111997 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.111976 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-876m9"] Jul 16 22:16:31.121565 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.121535 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw"] Jul 16 22:16:31.127235 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:31.127216 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-5d61a3-29737335-g5vqw"] Jul 16 22:16:32.675601 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:32.675549 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0450ce4-c13c-4e66-af92-b3011167f7bc" path="/var/lib/kubelet/pods/a0450ce4-c13c-4e66-af92-b3011167f7bc/volumes" Jul 16 22:16:32.675988 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:32.675811 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3c995b-32f6-4934-a9d1-a0bc562cbf90" path="/var/lib/kubelet/pods/ba3c995b-32f6-4934-a9d1-a0bc562cbf90/volumes" Jul 16 22:16:32.676031 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:16:32.675995 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e0aa37-d000-476c-bf5f-622d7f9ab616" path="/var/lib/kubelet/pods/d9e0aa37-d000-476c-bf5f-622d7f9ab616/volumes" Jul 16 22:18:54.724177 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:18:54.724146 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:18:54.731911 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:18:54.731887 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:19:46.178051 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.178017 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4"] Jul 16 22:19:46.178568 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.178227 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" podUID="8dcb7973-6dd9-442b-97da-5ce6b0b56481" containerName="maas-api" containerID="cri-o://16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34" gracePeriod=30 Jul 16 22:19:46.427954 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.427929 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:19:46.554030 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.553990 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kltqm\" (UniqueName: \"kubernetes.io/projected/8dcb7973-6dd9-442b-97da-5ce6b0b56481-kube-api-access-kltqm\") pod \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " Jul 16 22:19:46.554208 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.554076 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls\") pod \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " Jul 16 22:19:46.554208 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.554181 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcb7973-6dd9-442b-97da-5ce6b0b56481-openshift-service-ca\") pod \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\" (UID: \"8dcb7973-6dd9-442b-97da-5ce6b0b56481\") " Jul 16 22:19:46.554528 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.554503 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dcb7973-6dd9-442b-97da-5ce6b0b56481-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dcb7973-6dd9-442b-97da-5ce6b0b56481" (UID: "8dcb7973-6dd9-442b-97da-5ce6b0b56481"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:19:46.556428 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.556401 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dcb7973-6dd9-442b-97da-5ce6b0b56481-kube-api-access-kltqm" (OuterVolumeSpecName: "kube-api-access-kltqm") pod "8dcb7973-6dd9-442b-97da-5ce6b0b56481" (UID: "8dcb7973-6dd9-442b-97da-5ce6b0b56481"). InnerVolumeSpecName "kube-api-access-kltqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:19:46.556541 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.556437 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dcb7973-6dd9-442b-97da-5ce6b0b56481" (UID: "8dcb7973-6dd9-442b-97da-5ce6b0b56481"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:19:46.655131 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.655097 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kltqm\" (UniqueName: \"kubernetes.io/projected/8dcb7973-6dd9-442b-97da-5ce6b0b56481-kube-api-access-kltqm\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:19:46.655131 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.655125 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcb7973-6dd9-442b-97da-5ce6b0b56481-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:19:46.655131 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:46.655135 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcb7973-6dd9-442b-97da-5ce6b0b56481-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:19:47.011289 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.011255 2578 generic.go:356] "Generic (PLEG): container finished" podID="8dcb7973-6dd9-442b-97da-5ce6b0b56481" containerID="16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34" exitCode=0 Jul 16 22:19:47.011448 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.011312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" event={"ID":"8dcb7973-6dd9-442b-97da-5ce6b0b56481","Type":"ContainerDied","Data":"16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34"} Jul 16 22:19:47.011448 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.011359 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" event={"ID":"8dcb7973-6dd9-442b-97da-5ce6b0b56481","Type":"ContainerDied","Data":"fb1df0586f8eb078ca38a1b7b57ecada5e97c7ac2b5312eff40da8221e160468"} Jul 16 22:19:47.011448 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.011379 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4" Jul 16 22:19:47.011634 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.011382 2578 scope.go:117] "RemoveContainer" containerID="16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34" Jul 16 22:19:47.020548 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.020529 2578 scope.go:117] "RemoveContainer" containerID="16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34" Jul 16 22:19:47.020816 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:19:47.020797 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34\": container with ID starting with 16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34 not found: ID does not exist" containerID="16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34" Jul 16 22:19:47.020866 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.020826 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34"} err="failed to get container status \"16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34\": rpc error: code = NotFound desc = could not find container \"16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34\": container with ID starting with 16d441a0e16f695527b55c5c48fd505f10257afc8f1bfb9b414bf0e4efc28f34 not found: ID does not exist" Jul 16 22:19:47.029726 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.029704 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4"] Jul 16 22:19:47.037579 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:47.037539 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5d61a3-6cf8d75d9f-2vvn4"] Jul 16 22:19:48.679807 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:48.679777 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dcb7973-6dd9-442b-97da-5ce6b0b56481" path="/var/lib/kubelet/pods/8dcb7973-6dd9-442b-97da-5ce6b0b56481/volumes" Jul 16 22:19:53.273923 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.273894 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4"] Jul 16 22:19:53.274302 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.274116 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" podUID="6be80e4b-9a72-48e4-aee5-058aa6c54598" containerName="maas-api" containerID="cri-o://a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e" gracePeriod=30 Jul 16 22:19:53.525596 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.525519 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:19:53.612856 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.612828 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls\") pod \"6be80e4b-9a72-48e4-aee5-058aa6c54598\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " Jul 16 22:19:53.613022 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.612866 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6be80e4b-9a72-48e4-aee5-058aa6c54598-openshift-service-ca\") pod \"6be80e4b-9a72-48e4-aee5-058aa6c54598\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " Jul 16 22:19:53.613022 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.612897 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fqtt\" (UniqueName: \"kubernetes.io/projected/6be80e4b-9a72-48e4-aee5-058aa6c54598-kube-api-access-5fqtt\") pod \"6be80e4b-9a72-48e4-aee5-058aa6c54598\" (UID: \"6be80e4b-9a72-48e4-aee5-058aa6c54598\") " Jul 16 22:19:53.613284 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.613256 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be80e4b-9a72-48e4-aee5-058aa6c54598-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6be80e4b-9a72-48e4-aee5-058aa6c54598" (UID: "6be80e4b-9a72-48e4-aee5-058aa6c54598"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 22:19:53.615184 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.615151 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be80e4b-9a72-48e4-aee5-058aa6c54598-kube-api-access-5fqtt" (OuterVolumeSpecName: "kube-api-access-5fqtt") pod "6be80e4b-9a72-48e4-aee5-058aa6c54598" (UID: "6be80e4b-9a72-48e4-aee5-058aa6c54598"). InnerVolumeSpecName "kube-api-access-5fqtt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:19:53.615306 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.615222 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6be80e4b-9a72-48e4-aee5-058aa6c54598" (UID: "6be80e4b-9a72-48e4-aee5-058aa6c54598"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:19:53.714074 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.714045 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6be80e4b-9a72-48e4-aee5-058aa6c54598-maas-api-tls\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:19:53.714074 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.714070 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6be80e4b-9a72-48e4-aee5-058aa6c54598-openshift-service-ca\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:19:53.714222 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:53.714085 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fqtt\" (UniqueName: \"kubernetes.io/projected/6be80e4b-9a72-48e4-aee5-058aa6c54598-kube-api-access-5fqtt\") on node \"ip-10-0-136-44.ec2.internal\" DevicePath \"\"" Jul 16 22:19:54.045731 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.045694 2578 generic.go:356] "Generic (PLEG): container finished" podID="6be80e4b-9a72-48e4-aee5-058aa6c54598" containerID="a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e" exitCode=0 Jul 16 22:19:54.045936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.045746 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" Jul 16 22:19:54.045936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.045784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" event={"ID":"6be80e4b-9a72-48e4-aee5-058aa6c54598","Type":"ContainerDied","Data":"a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e"} Jul 16 22:19:54.045936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.045840 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4" event={"ID":"6be80e4b-9a72-48e4-aee5-058aa6c54598","Type":"ContainerDied","Data":"cf9c4e1199f4a74741cf6fdfc36c06b9885e7e9997219539cfd1123414b68a5b"} Jul 16 22:19:54.045936 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.045861 2578 scope.go:117] "RemoveContainer" containerID="a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e" Jul 16 22:19:54.055549 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.055532 2578 scope.go:117] "RemoveContainer" containerID="a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e" Jul 16 22:19:54.055829 ip-10-0-136-44 kubenswrapper[2578]: E0716 22:19:54.055810 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e\": container with ID starting with a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e not found: ID does not exist" containerID="a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e" Jul 16 22:19:54.055883 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.055840 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e"} err="failed to get container status \"a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e\": rpc error: code = NotFound desc = could not find container \"a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e\": container with ID starting with a8eede081e9336ba5af920878bb0944f561ac7764aab1bde8694623f2ec0a20e not found: ID does not exist" Jul 16 22:19:54.066377 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.066347 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4"] Jul 16 22:19:54.072730 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.072705 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ff5f80-7c9d5dd8b7-sx8v4"] Jul 16 22:19:54.676238 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:54.676204 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be80e4b-9a72-48e4-aee5-058aa6c54598" path="/var/lib/kubelet/pods/6be80e4b-9a72-48e4-aee5-058aa6c54598/volumes" Jul 16 22:19:59.726482 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:19:59.726439 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d49c6f95d-wsv56_b330e71f-4a8d-448e-8cfa-03baf846f2a9/authorino/0.log" Jul 16 22:20:04.643956 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:04.643920 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b5ff8896-t9kx4_85271990-63ad-4be6-92fe-e4a41a5bdd4d/manager/0.log" Jul 16 22:20:05.110110 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:05.110073 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db5748bcb-h8d6b_c62bb8eb-c617-4c20-a61e-18befaa656e4/manager/0.log" Jul 16 22:20:05.118631 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:05.118603 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db5748bcb-h8d6b_c62bb8eb-c617-4c20-a61e-18befaa656e4/copy-manifests/0.log" Jul 16 22:20:05.564187 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:05.564156 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68fc5bb9d9-fkxsm_ddff45b9-2544-48a0-89ce-624c6cda6c29/maas-api/0.log" Jul 16 22:20:05.675301 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:05.675280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7z7x5_9d4e3106-8630-4535-8a94-16b8e2f53871/postgres/0.log" Jul 16 22:20:06.970730 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:06.970698 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d49c6f95d-wsv56_b330e71f-4a8d-448e-8cfa-03baf846f2a9/authorino/0.log" Jul 16 22:20:08.233461 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:08.233431 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-czrgb_25430bde-92fd-4591-a832-cd5df76d33d6/discovery/0.log" Jul 16 22:20:08.689635 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:08.689603 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-2vrvh_4de09e9c-9c6b-4a95-90ca-93d524f9c519/payload-pre-processing/0.log" Jul 16 22:20:08.924284 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:08.924261 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6575ff56fb-h4b7b_ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0/router/0.log" Jul 16 22:20:09.260573 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:09.260536 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dndks_b4176a13-a19f-4659-ae18-128b50872e38/main/0.log" Jul 16 22:20:09.367676 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:09.367649 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-z68wv_77fc7706-fcc6-4944-a37b-45c02c0884fb/main/0.log" Jul 16 22:20:16.848860 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:16.848830 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ddq22_853ab9b3-3a12-4858-a741-abb04892dd5b/global-pull-secret-syncer/0.log" Jul 16 22:20:17.015977 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:17.015946 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rdcql_1b264886-0acf-4405-b7c4-e352f58819aa/konnectivity-agent/0.log" Jul 16 22:20:17.070810 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:17.070789 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-44.ec2.internal_8fa8481daebd4056943f9c05c427dbe3/haproxy/0.log" Jul 16 22:20:22.696580 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:22.696524 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d49c6f95d-wsv56_b330e71f-4a8d-448e-8cfa-03baf846f2a9/authorino/0.log" Jul 16 22:20:24.751061 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:24.750976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-xzmgw_192d89ab-ec22-4974-a8b2-01f6eb9778fd/cluster-monitoring-operator/0.log" Jul 16 22:20:25.106297 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:25.106271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nhmnb_f7228962-c722-49df-93a4-de1a9f880d97/node-exporter/0.log" Jul 16 22:20:25.126600 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:25.126577 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nhmnb_f7228962-c722-49df-93a4-de1a9f880d97/kube-rbac-proxy/0.log" Jul 16 22:20:25.152748 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:25.152726 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nhmnb_f7228962-c722-49df-93a4-de1a9f880d97/init-textfile/0.log" Jul 16 22:20:26.863064 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863034 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np"] Jul 16 22:20:26.863503 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863486 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be80e4b-9a72-48e4-aee5-058aa6c54598" containerName="maas-api" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863505 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be80e4b-9a72-48e4-aee5-058aa6c54598" containerName="maas-api" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863521 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3c995b-32f6-4934-a9d1-a0bc562cbf90" containerName="cleanup" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863527 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3c995b-32f6-4934-a9d1-a0bc562cbf90" containerName="cleanup" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863576 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e0aa37-d000-476c-bf5f-622d7f9ab616" containerName="cleanup" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863583 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e0aa37-d000-476c-bf5f-622d7f9ab616" containerName="cleanup" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863592 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dcb7973-6dd9-442b-97da-5ce6b0b56481" containerName="maas-api" Jul 16 22:20:26.863597 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863597 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dcb7973-6dd9-442b-97da-5ce6b0b56481" containerName="maas-api" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863604 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0450ce4-c13c-4e66-af92-b3011167f7bc" containerName="cleanup" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863609 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0450ce4-c13c-4e66-af92-b3011167f7bc" containerName="cleanup" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863685 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be80e4b-9a72-48e4-aee5-058aa6c54598" containerName="maas-api" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863692 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e0aa37-d000-476c-bf5f-622d7f9ab616" containerName="cleanup" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863703 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dcb7973-6dd9-442b-97da-5ce6b0b56481" containerName="maas-api" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863708 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba3c995b-32f6-4934-a9d1-a0bc562cbf90" containerName="cleanup" Jul 16 22:20:26.863818 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.863714 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0450ce4-c13c-4e66-af92-b3011167f7bc" containerName="cleanup" Jul 16 22:20:26.866965 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.866947 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.869650 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.869630 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qvtkv\"/\"default-dockercfg-bx2pd\"" Jul 16 22:20:26.869838 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.869652 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qvtkv\"/\"kube-root-ca.crt\"" Jul 16 22:20:26.870471 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.870456 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qvtkv\"/\"openshift-service-ca.crt\"" Jul 16 22:20:26.875109 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.875088 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np"] Jul 16 22:20:26.887324 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.887294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-podres\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.887515 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.887345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrwl5\" (UniqueName: \"kubernetes.io/projected/c7111452-5fe6-4bda-8815-003246a30744-kube-api-access-mrwl5\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.887515 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.887421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-sys\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.887515 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.887451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-proc\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.887515 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.887482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-lib-modules\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.987913 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.987877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-sys\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.987913 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.987913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-proc\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.987939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-lib-modules\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.987975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-podres\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.988003 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-sys\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.988019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrwl5\" (UniqueName: \"kubernetes.io/projected/c7111452-5fe6-4bda-8815-003246a30744-kube-api-access-mrwl5\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.988004 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-proc\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.988096 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-lib-modules\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.988140 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.988129 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c7111452-5fe6-4bda-8815-003246a30744-podres\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:26.996822 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:26.996792 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrwl5\" (UniqueName: \"kubernetes.io/projected/c7111452-5fe6-4bda-8815-003246a30744-kube-api-access-mrwl5\") pod \"perf-node-gather-daemonset-jq7np\" (UID: \"c7111452-5fe6-4bda-8815-003246a30744\") " pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:27.032494 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:27.032463 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-t25mv_293491f1-a543-4ae5-a8f9-ccdf2cd8957b/networking-console-plugin/0.log" Jul 16 22:20:27.177515 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:27.177423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:27.309620 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:27.309592 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np"] Jul 16 22:20:27.311763 ip-10-0-136-44 kubenswrapper[2578]: W0716 22:20:27.311741 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc7111452_5fe6_4bda_8815_003246a30744.slice/crio-c377404e5beb25f129378a1daa761457d9747c765faf945884c479cafe93d44b WatchSource:0}: Error finding container c377404e5beb25f129378a1daa761457d9747c765faf945884c479cafe93d44b: Status 404 returned error can't find the container with id c377404e5beb25f129378a1daa761457d9747c765faf945884c479cafe93d44b Jul 16 22:20:27.649731 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:27.649704 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/1.log" Jul 16 22:20:27.657571 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:27.657519 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s4d5p_b048d364-094e-443b-a259-7d836250a940/console-operator/2.log" Jul 16 22:20:28.166143 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:28.166113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-crmdz_bfa93aad-a92d-4020-82a2-53541ba23065/download-server/0.log" Jul 16 22:20:28.198464 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:28.198436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" event={"ID":"c7111452-5fe6-4bda-8815-003246a30744","Type":"ContainerStarted","Data":"fae6ef9c181acdb00cbfb8a46d9a1b72ba7d8a0af45de523add11897da49e9d2"} Jul 16 22:20:28.198678 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:28.198490 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" event={"ID":"c7111452-5fe6-4bda-8815-003246a30744","Type":"ContainerStarted","Data":"c377404e5beb25f129378a1daa761457d9747c765faf945884c479cafe93d44b"} Jul 16 22:20:28.198678 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:28.198512 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:28.215606 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:28.215541 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" podStartSLOduration=2.215526734 podStartE2EDuration="2.215526734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:20:28.213431717 +0000 UTC m=+2494.014318379" watchObservedRunningTime="2026-07-16 22:20:28.215526734 +0000 UTC m=+2494.016413385" Jul 16 22:20:28.729931 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:28.729893 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-2kpcw_b9326bb8-a637-44ec-b922-ab4f9f87a429/volume-data-source-validator/0.log" Jul 16 22:20:29.812714 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:29.812678 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cv7nc_f6571ef5-21df-43b3-b213-dd81d521d969/dns/0.log" Jul 16 22:20:29.835611 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:29.835585 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cv7nc_f6571ef5-21df-43b3-b213-dd81d521d969/kube-rbac-proxy/0.log" Jul 16 22:20:29.960356 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:29.960326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pjkrj_4a85dd8e-53c8-408d-bd91-1a46d81a43fc/dns-node-resolver/0.log" Jul 16 22:20:30.570111 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:30.570077 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd4s2_f870e849-9509-4e3a-9264-0ec11181dfb4/node-ca/0.log" Jul 16 22:20:31.701612 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:31.701577 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-czrgb_25430bde-92fd-4591-a832-cd5df76d33d6/discovery/0.log" Jul 16 22:20:31.869177 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:31.869142 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-2vrvh_4de09e9c-9c6b-4a95-90ca-93d524f9c519/payload-pre-processing/0.log" Jul 16 22:20:31.932503 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:31.932476 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6575ff56fb-h4b7b_ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0/router/0.log" Jul 16 22:20:32.663273 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:32.663242 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-slv9x_59fe218f-3f52-4ddb-b052-762f1e9ddef8/serve-healthcheck-canary/0.log" Jul 16 22:20:33.122241 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:33.122209 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-62gc6_b078fc92-aee9-4bf5-a3b6-bce92c892808/insights-operator/0.log" Jul 16 22:20:33.125687 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:33.125667 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-62gc6_b078fc92-aee9-4bf5-a3b6-bce92c892808/insights-operator/1.log" Jul 16 22:20:33.294137 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:33.294105 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rzkg4_e93aebfb-7224-44b2-a908-43b5ef7d7897/kube-rbac-proxy/0.log" Jul 16 22:20:33.318361 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:33.318337 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rzkg4_e93aebfb-7224-44b2-a908-43b5ef7d7897/exporter/0.log" Jul 16 22:20:33.342654 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:33.342626 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rzkg4_e93aebfb-7224-44b2-a908-43b5ef7d7897/extractor/0.log" Jul 16 22:20:34.213694 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:34.213662 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qvtkv/perf-node-gather-daemonset-jq7np" Jul 16 22:20:35.860769 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:35.860733 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b5ff8896-t9kx4_85271990-63ad-4be6-92fe-e4a41a5bdd4d/manager/0.log" Jul 16 22:20:36.161600 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:36.161549 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db5748bcb-h8d6b_c62bb8eb-c617-4c20-a61e-18befaa656e4/manager/0.log" Jul 16 22:20:36.179633 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:36.179610 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db5748bcb-h8d6b_c62bb8eb-c617-4c20-a61e-18befaa656e4/copy-manifests/0.log" Jul 16 22:20:37.426763 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:37.426731 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-czrgb_25430bde-92fd-4591-a832-cd5df76d33d6/discovery/0.log" Jul 16 22:20:37.520785 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:37.520740 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6844f9fcbb-4nnbj_9da3aac1-ee2a-4488-a610-f5e328e77804/manager/0.log" Jul 16 22:20:37.541233 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:37.541203 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-2vrvh_4de09e9c-9c6b-4a95-90ca-93d524f9c519/payload-pre-processing/0.log" Jul 16 22:20:37.585595 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:37.585549 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6575ff56fb-h4b7b_ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0/router/0.log" Jul 16 22:20:39.113027 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:39.112989 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-czrgb_25430bde-92fd-4591-a832-cd5df76d33d6/discovery/0.log" Jul 16 22:20:39.232046 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:39.232015 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-2vrvh_4de09e9c-9c6b-4a95-90ca-93d524f9c519/payload-pre-processing/0.log" Jul 16 22:20:39.282400 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:39.282367 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6575ff56fb-h4b7b_ce3df8b6-f0a0-4fc4-82be-5bd5dc5ee3e0/router/0.log" Jul 16 22:20:42.806153 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:42.806119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-5mx67_c6594bdb-a475-4973-8068-f6bf5ddd341a/kube-storage-version-migrator-operator/1.log" Jul 16 22:20:42.808259 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:42.808219 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-5mx67_c6594bdb-a475-4973-8068-f6bf5ddd341a/kube-storage-version-migrator-operator/0.log" Jul 16 22:20:43.779626 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:43.779587 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4wxqq_d5b49fd4-2713-41dc-97f8-e3a29f9e14c0/kube-multus/0.log" Jul 16 22:20:43.984627 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:43.984601 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sb58x_a7fc97df-84c8-45e9-af2f-1099dc537363/kube-multus-additional-cni-plugins/0.log" Jul 16 22:20:44.008226 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:44.008202 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sb58x_a7fc97df-84c8-45e9-af2f-1099dc537363/egress-router-binary-copy/0.log" Jul 16 22:20:44.030033 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:44.029950 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sb58x_a7fc97df-84c8-45e9-af2f-1099dc537363/cni-plugins/0.log" Jul 16 22:20:44.055966 ip-10-0-136-44 kubenswrapper[2578]: I0716 22:20:44.055935 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sb58x_a7fc97df-84c8-45e9-af2f-1099dc537363/bond-cni-plugin/0.log"