Jul 14 11:17:50.017541 ip-10-0-136-255 systemd[1]: Starting Kubernetes Kubelet... Jul 14 11:17:50.453145 ip-10-0-136-255 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:17:50.453145 ip-10-0-136-255 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 11:17:50.453145 ip-10-0-136-255 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:17:50.453145 ip-10-0-136-255 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 11:17:50.453145 ip-10-0-136-255 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:17:50.455072 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.454979 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462449 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462466 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462469 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462473 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462476 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462479 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462482 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:17:50.462474 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462484 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462487 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462490 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462493 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462495 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462500 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462503 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462507 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462510 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462513 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462516 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462519 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462522 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462525 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462527 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462530 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462532 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462534 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462537 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:17:50.462747 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462540 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462543 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462545 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462548 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462551 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462553 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462555 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462558 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462560 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462563 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462565 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462567 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462570 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462575 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462579 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462582 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462585 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462588 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462590 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462593 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:17:50.463202 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462595 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462598 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462600 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462603 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462605 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462608 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462610 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462612 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462615 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462617 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462620 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462622 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462628 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462632 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462634 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462637 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462639 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462641 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462644 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462646 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:17:50.463702 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462649 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462651 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462654 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462657 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462659 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462662 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462664 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462667 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462669 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462672 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462674 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462677 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462679 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462682 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462684 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462686 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462689 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462692 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462694 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462697 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:17:50.464170 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462699 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462701 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462703 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462706 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462709 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462711 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462713 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462716 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462718 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462720 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462723 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462725 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.462727 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463480 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463488 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463492 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463495 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463498 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463501 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463504 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:17:50.464764 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463506 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463510 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463514 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463516 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463519 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463522 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463526 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463529 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463532 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463534 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463538 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463541 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463544 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463546 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463548 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463552 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463554 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463557 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463559 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:17:50.465283 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463562 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463564 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463567 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463569 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463571 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463574 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463576 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463579 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463582 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463585 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463587 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463589 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463592 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463595 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463597 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463600 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463602 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463605 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463607 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463610 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:17:50.465759 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463613 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463615 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463617 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463621 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463627 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463630 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463632 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463635 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463637 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463640 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463642 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463645 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463647 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463649 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463652 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463654 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463657 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463659 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463662 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:17:50.466247 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463664 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463667 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463670 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463672 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463675 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463677 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463680 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463682 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463684 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463687 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463689 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463691 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463694 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463696 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463699 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463701 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463703 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463706 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463710 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463713 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:17:50.466731 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463715 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463720 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463723 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463725 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463728 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463730 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463733 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463735 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463738 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463740 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463743 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463745 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463747 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.463750 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464118 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464123 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464126 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464128 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464131 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464133 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:17:50.467231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464136 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464139 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464141 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464144 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464147 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464149 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464152 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464154 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464157 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464159 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464162 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464165 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464167 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464172 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464174 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464177 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464180 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464182 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464185 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464187 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:17:50.467729 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464189 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464192 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464194 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464197 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464199 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464203 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464206 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464208 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464211 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464213 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464215 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464218 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464220 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464223 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464225 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464227 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464230 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464233 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464235 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464238 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:17:50.468211 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464242 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464245 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464248 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464274 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464279 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464283 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464286 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464289 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464291 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464294 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464297 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464299 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464302 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464304 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464307 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464309 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464311 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464317 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464320 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:17:50.468706 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464323 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464328 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464331 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464334 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464336 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464339 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464342 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464345 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464347 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464349 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464352 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464354 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464357 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464359 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464362 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464364 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464367 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464369 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464373 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464376 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:17:50.469156 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464378 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464380 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464383 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464385 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464387 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464390 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464392 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464395 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464398 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464400 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464405 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464408 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464410 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.464413 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464484 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464491 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464497 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464502 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464506 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464509 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464513 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 11:17:50.469665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464517 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464520 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464523 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464526 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464529 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464532 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464535 2568 flags.go:64] FLAG: --cgroup-root="" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464538 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464541 2568 flags.go:64] FLAG: --client-ca-file="" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464545 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464548 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464552 2568 flags.go:64] FLAG: --cluster-domain="" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464555 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464558 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464561 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464564 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464567 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464570 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464573 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464576 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464579 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464584 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464586 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464589 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 11:17:50.470211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464592 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464596 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464599 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464601 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464604 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464607 2568 flags.go:64] FLAG: --enable-server="true" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464609 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464613 2568 flags.go:64] FLAG: --event-burst="100" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464616 2568 flags.go:64] FLAG: --event-qps="50" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464619 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464622 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464624 2568 flags.go:64] FLAG: --eviction-hard="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464628 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464631 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464634 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464636 2568 flags.go:64] FLAG: --eviction-soft="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464639 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464643 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464647 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464649 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464652 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464655 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464657 2568 flags.go:64] FLAG: --feature-gates="" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464661 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464664 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 11:17:50.470806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464666 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464669 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464672 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464675 2568 flags.go:64] FLAG: --help="false" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464680 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464682 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464685 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464688 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464691 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464694 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464697 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464699 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464702 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464705 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464707 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464710 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464713 2568 flags.go:64] FLAG: --kube-reserved="" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464716 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464718 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464721 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464723 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464726 2568 flags.go:64] FLAG: --lock-file="" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464729 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464732 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 11:17:50.471451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464739 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464744 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464747 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464750 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464753 2568 flags.go:64] FLAG: --logging-format="text" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464755 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464759 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464761 2568 flags.go:64] FLAG: --manifest-url="" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464765 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464769 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464771 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464776 2568 flags.go:64] FLAG: --max-pods="110" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464780 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464783 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464786 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464788 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464791 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464794 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464797 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464808 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464811 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464814 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464817 2568 flags.go:64] FLAG: --pod-cidr="" Jul 14 11:17:50.472375 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464820 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464824 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464827 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464830 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464833 2568 flags.go:64] FLAG: --port="10250" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464836 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464838 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07882d2cdf8f69546" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464841 2568 flags.go:64] FLAG: --qos-reserved="" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464844 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464847 2568 flags.go:64] FLAG: --register-node="true" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464850 2568 flags.go:64] FLAG: --register-with-taints="" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464853 2568 flags.go:64] FLAG: --registry-burst="10" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464856 2568 flags.go:64] FLAG: --registry-qps="5" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464859 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464862 2568 flags.go:64] FLAG: --reserved-memory="" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464865 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464868 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464871 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464873 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464876 2568 flags.go:64] FLAG: --runonce="false" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464879 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464882 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464885 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464888 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464891 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464894 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 11:17:50.473361 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464897 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464899 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464902 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464905 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464908 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464910 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464913 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464916 2568 flags.go:64] FLAG: --system-cgroups="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464918 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464923 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464926 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464928 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464932 2568 flags.go:64] FLAG: --tls-min-version="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464934 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464937 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464940 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464943 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464946 2568 flags.go:64] FLAG: --v="2" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464949 2568 flags.go:64] FLAG: --version="false" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464953 2568 flags.go:64] FLAG: --vmodule="" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464957 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.464960 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465070 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465074 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:17:50.474471 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465079 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465082 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465084 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465087 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465093 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465095 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465098 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465100 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465103 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465105 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465109 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465112 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465115 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465118 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465121 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465124 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465126 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465129 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465131 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:17:50.475072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465134 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465136 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465139 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465141 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465144 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465146 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465149 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465151 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465154 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465157 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465159 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465162 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465165 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465167 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465169 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465172 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465174 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465178 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465180 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465183 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:17:50.475825 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465186 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465188 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465191 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465193 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465196 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465198 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465201 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465203 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465206 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465208 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465210 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465213 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465215 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465218 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465220 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465222 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465225 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465227 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465230 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465233 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:17:50.476681 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465235 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465237 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465240 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465243 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465245 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465249 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465266 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465269 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465272 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465276 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465278 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465280 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465284 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465287 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465290 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465293 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465295 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465298 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465301 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:17:50.477174 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465304 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465306 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465309 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465311 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465314 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465316 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465319 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465321 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465324 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465327 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465329 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465331 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465334 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465337 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465339 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465341 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465344 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465347 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:17:50.477860 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.465349 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.466096 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.474291 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.474309 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.475236 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.475268 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475354 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475362 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475368 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475373 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475377 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475382 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475386 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475390 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475394 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:17:50.478643 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475398 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475402 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475406 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475410 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475415 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475420 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475424 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475428 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475432 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475436 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475440 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475444 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475448 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475452 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475457 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475461 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475465 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475469 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475474 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475478 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:17:50.479084 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475483 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475488 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475492 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475496 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475500 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475505 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475509 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475513 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475517 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475521 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475525 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475529 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475534 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475538 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475542 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475546 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475550 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475554 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475558 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475562 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:17:50.479833 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475566 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475570 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475574 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475578 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475582 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475586 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475591 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475594 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475601 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475607 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475612 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475617 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475621 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475626 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475631 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475638 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475643 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475648 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475652 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:17:50.480375 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475656 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475660 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475664 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475668 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475673 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475677 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475681 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475686 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475689 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475694 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475698 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475702 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475706 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475711 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475715 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475720 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475724 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475731 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475736 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475740 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:17:50.480835 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475745 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475749 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475753 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475758 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475762 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475767 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475771 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475775 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475779 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475783 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475788 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.475796 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475972 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475981 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475986 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:17:50.481385 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475990 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475995 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.475999 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476003 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476009 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476015 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476019 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476023 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476027 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476031 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476035 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476039 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476043 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476047 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476052 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476056 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476060 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476064 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476068 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476072 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:17:50.481750 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476076 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476081 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476085 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476090 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476094 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476099 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476103 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476107 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476111 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476116 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476120 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476124 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476128 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476132 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476136 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476140 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476143 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476148 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476152 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476156 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:17:50.482315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476160 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476164 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476168 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476172 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476176 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476180 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476184 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476188 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476192 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476196 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476200 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476205 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476209 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476213 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476217 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476221 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476225 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476229 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476234 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476239 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:17:50.482776 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476243 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476247 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476271 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476277 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476282 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476286 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476290 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476294 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476299 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476303 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476307 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476312 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476316 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476320 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476324 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476328 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476332 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476336 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476340 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:17:50.483303 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476344 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476349 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476354 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476358 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476362 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476366 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476370 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476374 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476378 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476382 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476386 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476390 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476395 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476402 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476407 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476412 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:17:50.483792 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:50.476416 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:17:50.484177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.476425 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:17:50.484177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.476567 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 11:17:50.484177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.479494 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 11:17:50.484177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.481231 2568 server.go:1019] "Starting client certificate rotation" Jul 14 11:17:50.484177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.481346 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 11:17:50.484177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.481381 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 11:17:50.485120 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.485103 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 11:17:50.490539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.490520 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 11:17:50.501526 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.501500 2568 log.go:25] "Validated CRI v1 runtime API" Jul 14 11:17:50.503714 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.503701 2568 log.go:25] "Validated CRI v1 image API" Jul 14 11:17:50.504550 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.504532 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 11:17:50.509043 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.509015 2568 fs.go:135] Filesystem UUIDs: map[1d246548-e4e1-4ebb-969c-a65d6869f145:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ceb2e500-bf8e-4838-98fd-2df6aaca634f:/dev/nvme0n1p4] Jul 14 11:17:50.509104 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.509041 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 11:17:50.514150 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.514018 2568 manager.go:216] Machine: {Timestamp:2026-07-14 11:17:50.512686252 +0000 UTC m=+0.368520418 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3082702 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2150845d75be4e282ac7d9b09f3ead SystemUUID:ec215084-5d75-be4e-282a-c7d9b09f3ead BootID:77107c56-d0fa-40d0-82ee-c8042340e3a4 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:b2:18:b2:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:b2:18:b2:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:51:1d:fa:51:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 11:17:50.514150 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.514144 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 11:17:50.514279 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.514224 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 11:17:50.517434 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.517419 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 11:17:50.517505 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.517477 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 11:17:50.517669 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.517514 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-255.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 11:17:50.518355 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.518345 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 11:17:50.518391 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.518357 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 11:17:50.518391 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.518365 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 11:17:50.519132 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.519121 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 11:17:50.520833 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.520823 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 11:17:50.524355 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524343 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 14 11:17:50.524474 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524465 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 11:17:50.524503 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524496 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 14 11:17:50.524530 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524505 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 11:17:50.524530 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524522 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 11:17:50.524588 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524531 2568 kubelet.go:398] "Adding apiserver pod source" Jul 14 11:17:50.524588 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524540 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 11:17:50.524925 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.524911 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 11:17:50.527356 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.527341 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 11:17:50.529728 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.529711 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 11:17:50.532226 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532210 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 11:17:50.532317 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532248 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 11:17:50.532532 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532504 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 11:17:50.532595 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532542 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 11:17:50.532595 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532557 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 11:17:50.532595 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532568 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 11:17:50.532595 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532581 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 11:17:50.532595 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.532589 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 11:17:50.533067 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533057 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 11:17:50.533125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533069 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 11:17:50.533125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533086 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 11:17:50.533125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533095 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 11:17:50.533125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533107 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 11:17:50.533249 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533132 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 11:17:50.533994 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533983 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 11:17:50.534052 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.533996 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 11:17:50.535028 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.535001 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-255.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 11:17:50.535028 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.534997 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 11:17:50.537534 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.537518 2568 server.go:1268] "Started kubelet" Jul 14 11:17:50.538405 ip-10-0-136-255 systemd[1]: Started Kubernetes Kubelet. Jul 14 11:17:50.538878 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.538661 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 11:17:50.538878 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.538648 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 11:17:50.538878 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.538727 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 11:17:50.540473 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.540443 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 11:17:50.540851 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.540838 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 14 11:17:50.545125 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.545092 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 11:17:50.546381 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.546362 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-255.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 11:17:50.549356 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.549342 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 11:17:50.549488 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.549465 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 11:17:50.549576 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.549489 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:50.550393 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550372 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 11:17:50.550393 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550395 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 11:17:50.550526 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550409 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 11:17:50.550583 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550549 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 14 11:17:50.550583 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550559 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 14 11:17:50.550910 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550884 2568 factory.go:55] Registering systemd factory Jul 14 11:17:50.551018 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.550927 2568 factory.go:223] Registration of the systemd container factory successfully Jul 14 11:17:50.551140 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.551128 2568 factory.go:153] Registering CRI-O factory Jul 14 11:17:50.551183 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.551142 2568 factory.go:223] Registration of the crio container factory successfully Jul 14 11:17:50.551215 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.551197 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 11:17:50.551280 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.551222 2568 factory.go:103] Registering Raw factory Jul 14 11:17:50.551280 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.551246 2568 manager.go:1196] Started watching for new ooms in manager Jul 14 11:17:50.551748 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.551735 2568 manager.go:349] Starting recovery of all containers Jul 14 11:17:50.555116 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.554866 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-255.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 11:17:50.555116 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.554885 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 11:17:50.556031 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.554956 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-255.ec2.internal.18c223e564c7c0c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-255.ec2.internal,UID:ip-10-0-136-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-255.ec2.internal,},FirstTimestamp:2026-07-14 11:17:50.537433288 +0000 UTC m=+0.393267451,LastTimestamp:2026-07-14 11:17:50.537433288 +0000 UTC m=+0.393267451,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-255.ec2.internal,}" Jul 14 11:17:50.563024 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.562947 2568 manager.go:354] Recovery completed Jul 14 11:17:50.567571 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.567557 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.569500 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.569481 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.569578 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.569511 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.569578 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.569520 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.569837 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.569818 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b7qn6" Jul 14 11:17:50.570100 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570087 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 11:17:50.570148 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570101 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 11:17:50.570148 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570115 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 14 11:17:50.570148 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570117 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.570816 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570801 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.570885 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570826 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.570885 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.570836 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.571174 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.571163 2568 policy_none.go:49] "None policy: Start" Jul 14 11:17:50.571207 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.571181 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 11:17:50.571207 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.571191 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 11:17:50.571936 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.571865 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-255.ec2.internal.18c223e566b105d3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-255.ec2.internal,UID:ip-10-0-136-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-255.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-255.ec2.internal,},FirstTimestamp:2026-07-14 11:17:50.569498067 +0000 UTC m=+0.425332230,LastTimestamp:2026-07-14 11:17:50.569498067 +0000 UTC m=+0.425332230,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-255.ec2.internal,}" Jul 14 11:17:50.572408 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.572391 2568 policy_none.go:47] "Start" Jul 14 11:17:50.578611 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.578594 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b7qn6" Jul 14 11:17:50.581914 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.581854 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-255.ec2.internal.18c223e566b14b1a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-255.ec2.internal,UID:ip-10-0-136-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-255.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-255.ec2.internal,},FirstTimestamp:2026-07-14 11:17:50.569515802 +0000 UTC m=+0.425349965,LastTimestamp:2026-07-14 11:17:50.569515802 +0000 UTC m=+0.425349965,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-255.ec2.internal,}" Jul 14 11:17:50.620899 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.620882 2568 manager.go:337] "Starting Device Plugin manager" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.620984 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.621000 2568 server.go:87] "Starting device plugin registration server" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.621244 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.621278 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.621374 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.621458 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.621469 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.622015 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 11:17:50.625092 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.622067 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:50.670307 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.670271 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 11:17:50.671544 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.671525 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 11:17:50.671653 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.671562 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 11:17:50.671653 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.671581 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 11:17:50.671653 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.671611 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 11:17:50.674596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.674579 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:17:50.722140 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.722078 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.723151 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.723136 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.723234 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.723163 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.723234 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.723172 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.723234 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.723196 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.734987 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.734962 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.735107 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.734990 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-255.ec2.internal\": node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:50.756191 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.756167 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:50.772177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.772156 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal"] Jul 14 11:17:50.772277 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.772211 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.773604 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.773587 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.773691 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.773622 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.773691 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.773637 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.774874 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.774859 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.774976 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.774961 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.775025 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.774994 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.775571 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.775556 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.775571 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.775577 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.775683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.775591 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.775683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.775608 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.775683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.775626 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.775683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.775635 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.776617 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.776602 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.776669 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.776629 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:50.777399 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.777383 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:50.777468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.777416 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:50.777468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.777430 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:50.801850 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.801828 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-255.ec2.internal\" not found" node="ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.806213 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.806196 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-255.ec2.internal\" not found" node="ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.852689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.852665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/525ade16321a29f4f96b69f63298cd87-config\") pod \"kube-apiserver-proxy-ip-10-0-136-255.ec2.internal\" (UID: \"525ade16321a29f4f96b69f63298cd87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.852764 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.852691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf7b9f377dc657d6e2281ba56c987b9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal\" (UID: \"bf7b9f377dc657d6e2281ba56c987b9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.852764 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.852710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf7b9f377dc657d6e2281ba56c987b9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal\" (UID: \"bf7b9f377dc657d6e2281ba56c987b9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.856527 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.856512 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:50.953586 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.953543 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf7b9f377dc657d6e2281ba56c987b9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal\" (UID: \"bf7b9f377dc657d6e2281ba56c987b9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.953586 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.953588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf7b9f377dc657d6e2281ba56c987b9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal\" (UID: \"bf7b9f377dc657d6e2281ba56c987b9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.953740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.953608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/525ade16321a29f4f96b69f63298cd87-config\") pod \"kube-apiserver-proxy-ip-10-0-136-255.ec2.internal\" (UID: \"525ade16321a29f4f96b69f63298cd87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.953740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.953656 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bf7b9f377dc657d6e2281ba56c987b9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal\" (UID: \"bf7b9f377dc657d6e2281ba56c987b9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.953740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.953669 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf7b9f377dc657d6e2281ba56c987b9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal\" (UID: \"bf7b9f377dc657d6e2281ba56c987b9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.953740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:50.953668 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/525ade16321a29f4f96b69f63298cd87-config\") pod \"kube-apiserver-proxy-ip-10-0-136-255.ec2.internal\" (UID: \"525ade16321a29f4f96b69f63298cd87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" Jul 14 11:17:50.956600 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:50.956584 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.057442 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.057373 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.103574 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.103549 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:51.109280 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.109244 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" Jul 14 11:17:51.158315 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.158288 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.258799 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.258761 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.359389 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.359309 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.443437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.443414 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:17:51.459740 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.459720 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.480893 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.480871 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 11:17:51.481009 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.480989 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 11:17:51.481045 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.480986 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 11:17:51.560326 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.560302 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.581965 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.581931 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 11:12:50 +0000 UTC" deadline="2028-04-16 21:05:37.236118794 +0000 UTC" Jul 14 11:17:51.581965 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.581960 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15417h47m45.654161364s" Jul 14 11:17:51.652394 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.652374 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:17:51.653926 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.653909 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:17:51.654015 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.653947 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:17:51.654015 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.653964 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:17:51.661311 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.661293 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.738413 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:51.738375 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7b9f377dc657d6e2281ba56c987b9b.slice/crio-3f3fc85e8c74539ccd95e11ce3ea385aa3eea0c4d1b13212698edba29c3a0c96 WatchSource:0}: Error finding container 3f3fc85e8c74539ccd95e11ce3ea385aa3eea0c4d1b13212698edba29c3a0c96: Status 404 returned error can't find the container with id 3f3fc85e8c74539ccd95e11ce3ea385aa3eea0c4d1b13212698edba29c3a0c96 Jul 14 11:17:51.740320 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:51.740298 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525ade16321a29f4f96b69f63298cd87.slice/crio-1e6b43070d535f31e36c31f7139478cc8f2dd6c4aafbd5c68ce14b8134b34e5d WatchSource:0}: Error finding container 1e6b43070d535f31e36c31f7139478cc8f2dd6c4aafbd5c68ce14b8134b34e5d: Status 404 returned error can't find the container with id 1e6b43070d535f31e36c31f7139478cc8f2dd6c4aafbd5c68ce14b8134b34e5d Jul 14 11:17:51.743651 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.743633 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:17:51.762394 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.762374 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.798909 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:51.798890 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:17:51.862792 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.862756 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:51.963387 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:51.963317 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:52.063422 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.063391 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-255.ec2.internal\" not found" Jul 14 11:17:52.073334 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.073299 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:17:52.150035 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.149996 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" Jul 14 11:17:52.165400 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.165366 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 11:17:52.166461 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.166440 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" Jul 14 11:17:52.176920 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.176889 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 11:17:52.526426 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.526384 2568 apiserver.go:52] "Watching apiserver" Jul 14 11:17:52.536169 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.536140 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 11:17:52.538418 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.537938 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-8ns2q","openshift-ovn-kubernetes/ovnkube-node-ngnhm","kube-system/konnectivity-agent-8rlv7","kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f","openshift-cluster-node-tuning-operator/tuned-mr8rv","openshift-dns/node-resolver-t2jvz","openshift-image-registry/node-ca-hxzbm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal","openshift-multus/multus-additional-cni-plugins-766fl","kube-system/global-pull-secret-syncer-2qzv4","openshift-multus/multus-9bq6b","openshift-multus/network-metrics-daemon-8dclj","openshift-network-diagnostics/network-check-target-tffnh"] Jul 14 11:17:52.540447 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.539931 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.541537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.541347 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.543644 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.543621 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.543767 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.543694 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.543813 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.543631 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gslqb\"" Jul 14 11:17:52.544418 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.543879 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 11:17:52.544418 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.544049 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.544570 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.544450 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.546359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.544800 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.546359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.545013 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.546359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.545545 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 11:17:52.546359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.545747 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zqx8c\"" Jul 14 11:17:52.546977 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.546951 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 11:17:52.547535 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.547510 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 11:17:52.548080 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.547773 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qv2md\"" Jul 14 11:17:52.548080 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.547795 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.548080 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.547984 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.548612 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.548589 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 11:17:52.548612 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.548610 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 11:17:52.548768 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.548692 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2btcc\"" Jul 14 11:17:52.548831 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.548789 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.548831 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.548793 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 11:17:52.549075 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.549048 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.551232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.550312 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.551232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.550424 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 11:17:52.551232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.550655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.551232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.551028 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dxzcl\"" Jul 14 11:17:52.555523 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.554326 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.555908 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.555891 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dhqpf\"" Jul 14 11:17:52.556068 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.556049 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.556127 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.556114 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.557503 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.556821 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.557503 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.556928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.557503 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.557022 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:17:52.559287 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.558389 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 11:17:52.559287 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.558410 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-st6tx\"" Jul 14 11:17:52.559287 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.559040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.559287 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.559117 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 11:17:52.560394 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.559616 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.560394 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.559779 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 11:17:52.560645 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.560626 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 11:17:52.560906 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.560887 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn86h\" (UniqueName: \"kubernetes.io/projected/511a3de3-f45f-4efe-899d-ec67102344ab-kube-api-access-jn86h\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561292 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysctl-d\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-slash\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561339 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-ovn\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561427 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561433 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561440 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86q65\"" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.561501 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49806619-4c59-4b80-b24f-22162de9877b-host\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561548 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49806619-4c59-4b80-b24f-22162de9877b-serviceca\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-cni-bin\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561638 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmx9g\" (UniqueName: \"kubernetes.io/projected/45c57b79-de41-4891-a513-7102b9c8ffc3-kube-api-access-wmx9g\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-socket-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e44be89-e8f1-41ce-94c6-ee178019b118-tmp\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysctl-conf\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561811 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 11:17:52.561873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561813 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-run-netns\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561941 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-tuning-conf-dir\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.561989 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-lib-modules\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/445082e1-8f80-4e41-8164-2a8ce6a18b10-hosts-file\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-node-log\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-systemd\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562224 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-etc-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562261 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-host\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445082e1-8f80-4e41-8164-2a8ce6a18b10-tmp-dir\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-kubelet\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-registration-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/511a3de3-f45f-4efe-899d-ec67102344ab-host-slash\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-cnibin\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562507 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/296e9775-ef62-455d-9513-1fd1e4358764-agent-certs\") pod \"konnectivity-agent-8rlv7\" (UID: \"296e9775-ef62-455d-9513-1fd1e4358764\") " pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.562875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-systemd\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/511a3de3-f45f-4efe-899d-ec67102344ab-iptables-alerter-script\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-sys\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562733 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-var-lib-kubelet\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562733 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fst6j\"" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-var-lib-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562844 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-log-socket\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45c57b79-de41-4891-a513-7102b9c8ffc3-ovn-node-metrics-cert\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.562987 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563023 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/296e9775-ef62-455d-9513-1fd1e4358764-konnectivity-ca\") pod \"konnectivity-agent-8rlv7\" (UID: \"296e9775-ef62-455d-9513-1fd1e4358764\") " pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563049 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-systemd-units\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563025 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-env-overrides\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdrb4\" (UniqueName: \"kubernetes.io/projected/49806619-4c59-4b80-b24f-22162de9877b-kube-api-access-fdrb4\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-os-release\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-527ls\" (UniqueName: \"kubernetes.io/projected/eafa3614-4086-44e2-93e9-8839573a9397-kube-api-access-527ls\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-tuned\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.563647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563538 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vkpd\" (UniqueName: \"kubernetes.io/projected/445082e1-8f80-4e41-8164-2a8ce6a18b10-kube-api-access-6vkpd\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563570 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563605 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-ovnkube-config\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563610 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563633 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-cni-binary-copy\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-kubernetes\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.563713 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563788 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-cni-netd\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-modprobe-d\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-ovnkube-script-lib\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-device-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.563927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-system-cni-dir\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.564013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-run\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.564047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.564099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-sys-fs\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.564455 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.564138 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysconfig\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.565357 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.564225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk26h\" (UniqueName: \"kubernetes.io/projected/6e44be89-e8f1-41ce-94c6-ee178019b118-kube-api-access-dk26h\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.565357 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.564272 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x9p2\" (UniqueName: \"kubernetes.io/projected/deb191ef-3e9d-4ae6-afa1-84f41d02d051-kube-api-access-5x9p2\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.585466 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.585403 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hqzn2" Jul 14 11:17:52.594190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.594163 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hqzn2" Jul 14 11:17:52.652239 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.652216 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 11:17:52.665378 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.665529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-sys-fs\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.665529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysconfig\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.665529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk26h\" (UniqueName: \"kubernetes.io/projected/6e44be89-e8f1-41ce-94c6-ee178019b118-kube-api-access-dk26h\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.665529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e1eb7e2-1e05-4af6-93f1-894e406169c9-dbus\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.665529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665489 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-hostroot\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.665529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x9p2\" (UniqueName: \"kubernetes.io/projected/deb191ef-3e9d-4ae6-afa1-84f41d02d051-kube-api-access-5x9p2\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-os-release\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665542 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn86h\" (UniqueName: \"kubernetes.io/projected/511a3de3-f45f-4efe-899d-ec67102344ab-kube-api-access-jn86h\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysctl-d\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665636 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/442cf746-b413-40eb-906f-edac4e19044f-cni-binary-copy\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-slash\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-ovn\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49806619-4c59-4b80-b24f-22162de9877b-host\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49806619-4c59-4b80-b24f-22162de9877b-serviceca\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-kubelet\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.665834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-cni-bin\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-sys-fs\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmx9g\" (UniqueName: \"kubernetes.io/projected/45c57b79-de41-4891-a513-7102b9c8ffc3-kube-api-access-wmx9g\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-socket-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665932 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysconfig\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665938 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e44be89-e8f1-41ce-94c6-ee178019b118-tmp\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-socket-dir-parent\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.665998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-cni-multus\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysctl-conf\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-run-netns\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666078 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-tuning-conf-dir\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666142 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-lib-modules\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/445082e1-8f80-4e41-8164-2a8ce6a18b10-hosts-file\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666203 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-node-log\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-conf-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.666472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666274 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-systemd\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666299 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-etc-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-host\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666375 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-etc-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666537 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysctl-conf\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666604 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-run-netns\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666680 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666765 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-tuning-conf-dir\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-host\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666822 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-cni-bin\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.666955 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-lib-modules\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667009 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/445082e1-8f80-4e41-8164-2a8ce6a18b10-hosts-file\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667046 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-node-log\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-systemd\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667118 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49806619-4c59-4b80-b24f-22162de9877b-host\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667213 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-sysctl-d\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445082e1-8f80-4e41-8164-2a8ce6a18b10-tmp-dir\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667359 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-socket-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667441 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-slash\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667441 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-ovn\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-kubelet\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667465 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-kubelet\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667521 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-registration-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/511a3de3-f45f-4efe-899d-ec67102344ab-host-slash\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-cnibin\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667755 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/296e9775-ef62-455d-9513-1fd1e4358764-agent-certs\") pod \"konnectivity-agent-8rlv7\" (UID: \"296e9775-ef62-455d-9513-1fd1e4358764\") " pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667864 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49806619-4c59-4b80-b24f-22162de9877b-serviceca\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.667897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-systemd\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/511a3de3-f45f-4efe-899d-ec67102344ab-iptables-alerter-script\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-sys\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667962 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445082e1-8f80-4e41-8164-2a8ce6a18b10-tmp-dir\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-var-lib-kubelet\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.667929 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-registration-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668035 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668050 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-sys\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668088 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/511a3de3-f45f-4efe-899d-ec67102344ab-host-slash\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-system-cni-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668176 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-systemd\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668216 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-cnibin\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.668689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/511a3de3-f45f-4efe-899d-ec67102344ab-iptables-alerter-script\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-cni-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668731 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-netns\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-var-lib-kubelet\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-var-lib-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-log-socket\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668913 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-var-lib-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.668976 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-log-socket\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669012 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45c57b79-de41-4891-a513-7102b9c8ffc3-ovn-node-metrics-cert\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-cnibin\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669097 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669242 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669245 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669302 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/296e9775-ef62-455d-9513-1fd1e4358764-konnectivity-ca\") pod \"konnectivity-agent-8rlv7\" (UID: \"296e9775-ef62-455d-9513-1fd1e4358764\") " pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-cni-bin\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-multus-certs\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-systemd-units\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669411 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-env-overrides\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669503 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdrb4\" (UniqueName: \"kubernetes.io/projected/49806619-4c59-4b80-b24f-22162de9877b-kube-api-access-fdrb4\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669550 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-os-release\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-527ls\" (UniqueName: \"kubernetes.io/projected/eafa3614-4086-44e2-93e9-8839573a9397-kube-api-access-527ls\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669486 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-systemd-units\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669670 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-os-release\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-tuned\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e1eb7e2-1e05-4af6-93f1-894e406169c9-kubelet-config\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-env-overrides\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vkpd\" (UniqueName: \"kubernetes.io/projected/445082e1-8f80-4e41-8164-2a8ce6a18b10-kube-api-access-6vkpd\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.669890 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/296e9775-ef62-455d-9513-1fd1e4358764-konnectivity-ca\") pod \"konnectivity-agent-8rlv7\" (UID: \"296e9775-ef62-455d-9513-1fd1e4358764\") " pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-ovnkube-config\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.669971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-cni-binary-copy\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-kubernetes\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-etc-kubernetes\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670032 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-run-openvswitch\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670055 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670085 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-cni-netd\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-modprobe-d\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670132 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-ovnkube-script-lib\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-kubernetes\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670199 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670200 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45c57b79-de41-4891-a513-7102b9c8ffc3-host-cni-netd\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670155 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-device-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670271 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-modprobe-d\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670284 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-system-cni-dir\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.670664 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670345 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eafa3614-4086-44e2-93e9-8839573a9397-system-cni-dir\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670388 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/deb191ef-3e9d-4ae6-afa1-84f41d02d051-device-dir\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eafa3614-4086-44e2-93e9-8839573a9397-cni-binary-copy\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-run\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsxcm\" (UniqueName: \"kubernetes.io/projected/ecc55306-170e-463f-b3e5-263e3201fc1c-kube-api-access-tsxcm\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670576 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-k8s-cni-cncf-io\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670592 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/442cf746-b413-40eb-906f-edac4e19044f-multus-daemon-config\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6k8\" (UniqueName: \"kubernetes.io/projected/442cf746-b413-40eb-906f-edac4e19044f-kube-api-access-ht6k8\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670681 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e44be89-e8f1-41ce-94c6-ee178019b118-run\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670823 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-ovnkube-script-lib\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.670869 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45c57b79-de41-4891-a513-7102b9c8ffc3-ovnkube-config\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.671282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.671028 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/296e9775-ef62-455d-9513-1fd1e4358764-agent-certs\") pod \"konnectivity-agent-8rlv7\" (UID: \"296e9775-ef62-455d-9513-1fd1e4358764\") " pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.673641 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.673616 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e44be89-e8f1-41ce-94c6-ee178019b118-etc-tuned\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.674321 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.674298 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e44be89-e8f1-41ce-94c6-ee178019b118-tmp\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.674685 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.674662 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45c57b79-de41-4891-a513-7102b9c8ffc3-ovn-node-metrics-cert\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.678268 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.678242 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmx9g\" (UniqueName: \"kubernetes.io/projected/45c57b79-de41-4891-a513-7102b9c8ffc3-kube-api-access-wmx9g\") pod \"ovnkube-node-ngnhm\" (UID: \"45c57b79-de41-4891-a513-7102b9c8ffc3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.678514 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.678452 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" event={"ID":"525ade16321a29f4f96b69f63298cd87","Type":"ContainerStarted","Data":"1e6b43070d535f31e36c31f7139478cc8f2dd6c4aafbd5c68ce14b8134b34e5d"} Jul 14 11:17:52.680430 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.680407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk26h\" (UniqueName: \"kubernetes.io/projected/6e44be89-e8f1-41ce-94c6-ee178019b118-kube-api-access-dk26h\") pod \"tuned-mr8rv\" (UID: \"6e44be89-e8f1-41ce-94c6-ee178019b118\") " pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.680551 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.680460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" event={"ID":"bf7b9f377dc657d6e2281ba56c987b9b","Type":"ContainerStarted","Data":"3f3fc85e8c74539ccd95e11ce3ea385aa3eea0c4d1b13212698edba29c3a0c96"} Jul 14 11:17:52.686507 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.686463 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vkpd\" (UniqueName: \"kubernetes.io/projected/445082e1-8f80-4e41-8164-2a8ce6a18b10-kube-api-access-6vkpd\") pod \"node-resolver-t2jvz\" (UID: \"445082e1-8f80-4e41-8164-2a8ce6a18b10\") " pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.686983 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.686960 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn86h\" (UniqueName: \"kubernetes.io/projected/511a3de3-f45f-4efe-899d-ec67102344ab-kube-api-access-jn86h\") pod \"iptables-alerter-8ns2q\" (UID: \"511a3de3-f45f-4efe-899d-ec67102344ab\") " pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.687260 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.687231 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-527ls\" (UniqueName: \"kubernetes.io/projected/eafa3614-4086-44e2-93e9-8839573a9397-kube-api-access-527ls\") pod \"multus-additional-cni-plugins-766fl\" (UID: \"eafa3614-4086-44e2-93e9-8839573a9397\") " pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.687663 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.687644 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x9p2\" (UniqueName: \"kubernetes.io/projected/deb191ef-3e9d-4ae6-afa1-84f41d02d051-kube-api-access-5x9p2\") pod \"aws-ebs-csi-driver-node-ltm2f\" (UID: \"deb191ef-3e9d-4ae6-afa1-84f41d02d051\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.689067 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.688749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdrb4\" (UniqueName: \"kubernetes.io/projected/49806619-4c59-4b80-b24f-22162de9877b-kube-api-access-fdrb4\") pod \"node-ca-hxzbm\" (UID: \"49806619-4c59-4b80-b24f-22162de9877b\") " pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.771590 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771555 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-socket-dir-parent\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.771590 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-cni-multus\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.771845 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771716 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-socket-dir-parent\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.771845 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-cni-multus\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.771845 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-conf-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.771845 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771819 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771875 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-system-cni-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-cni-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-netns\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771969 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-cnibin\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.771980 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:52.772017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-cni-bin\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-multus-certs\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.772063 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:17:53.272035009 +0000 UTC m=+3.127869180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772065 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-cni-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-multus-certs\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e1eb7e2-1e05-4af6-93f1-894e406169c9-kubelet-config\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772124 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-netns\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-system-cni-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.771870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-multus-conf-dir\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772139 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-etc-kubernetes\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-cnibin\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772171 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-etc-kubernetes\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsxcm\" (UniqueName: \"kubernetes.io/projected/ecc55306-170e-463f-b3e5-263e3201fc1c-kube-api-access-tsxcm\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772186 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-cni-bin\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772232 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e1eb7e2-1e05-4af6-93f1-894e406169c9-kubelet-config\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.772244 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.771980 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.772305 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:17:53.272269922 +0000 UTC m=+3.128104075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-k8s-cni-cncf-io\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/442cf746-b413-40eb-906f-edac4e19044f-multus-daemon-config\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772416 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6k8\" (UniqueName: \"kubernetes.io/projected/442cf746-b413-40eb-906f-edac4e19044f-kube-api-access-ht6k8\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e1eb7e2-1e05-4af6-93f1-894e406169c9-dbus\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-hostroot\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-os-release\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/442cf746-b413-40eb-906f-edac4e19044f-cni-binary-copy\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-kubelet\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:52.773392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.773049 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/442cf746-b413-40eb-906f-edac4e19044f-multus-daemon-config\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.773099 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e1eb7e2-1e05-4af6-93f1-894e406169c9-dbus\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:52.773392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.773134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-os-release\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.772447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-run-k8s-cni-cncf-io\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.773177 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-host-var-lib-kubelet\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.773190 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/442cf746-b413-40eb-906f-edac4e19044f-hostroot\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.773606 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.773552 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/442cf746-b413-40eb-906f-edac4e19044f-cni-binary-copy\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.779192 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.779118 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:17:52.779192 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.779143 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:17:52.779192 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.779164 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:52.779449 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:52.779220 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:17:53.279202696 +0000 UTC m=+3.135036851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:52.782334 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.782309 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsxcm\" (UniqueName: \"kubernetes.io/projected/ecc55306-170e-463f-b3e5-263e3201fc1c-kube-api-access-tsxcm\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:52.782451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.782362 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6k8\" (UniqueName: \"kubernetes.io/projected/442cf746-b413-40eb-906f-edac4e19044f-kube-api-access-ht6k8\") pod \"multus-9bq6b\" (UID: \"442cf746-b413-40eb-906f-edac4e19044f\") " pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.861934 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.861892 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" Jul 14 11:17:52.871512 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.871472 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hxzbm" Jul 14 11:17:52.880778 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.880595 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-766fl" Jul 14 11:17:52.888511 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.888482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:17:52.895472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.895445 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" Jul 14 11:17:52.904628 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.904595 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t2jvz" Jul 14 11:17:52.912772 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.912751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:17:52.921320 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.921295 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8ns2q" Jul 14 11:17:52.928518 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.928496 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9bq6b" Jul 14 11:17:52.960542 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:52.960507 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:17:53.276704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.276630 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:53.276704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.276666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:53.276884 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.276799 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:53.276884 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.276801 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:53.276884 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.276866 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:17:54.276847559 +0000 UTC m=+4.132681710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:53.276991 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.276886 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:17:54.276877419 +0000 UTC m=+4.132711570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:53.318557 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.318525 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:17:53.377794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.377755 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:53.377975 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.377953 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:17:53.378020 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.377981 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:17:53.378020 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.377991 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:53.378124 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:53.378070 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:17:54.37803127 +0000 UTC m=+4.233865444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:53.596476 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.596392 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:12:52 +0000 UTC" deadline="2028-04-23 19:31:00.433130682 +0000 UTC" Jul 14 11:17:53.596476 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.596425 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15584h13m6.836710182s" Jul 14 11:17:53.678354 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.678106 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb191ef_3e9d_4ae6_afa1_84f41d02d051.slice/crio-7f2ad8e3df32030ad75655817262841feedb20974de75e3b72f67dd6d52723a4 WatchSource:0}: Error finding container 7f2ad8e3df32030ad75655817262841feedb20974de75e3b72f67dd6d52723a4: Status 404 returned error can't find the container with id 7f2ad8e3df32030ad75655817262841feedb20974de75e3b72f67dd6d52723a4 Jul 14 11:17:53.679231 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.679169 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49806619_4c59_4b80_b24f_22162de9877b.slice/crio-a2d2a75ff4b82f08c13fe2bc0b8ba9505b44b6a5291809de59fd14ae0134ac8a WatchSource:0}: Error finding container a2d2a75ff4b82f08c13fe2bc0b8ba9505b44b6a5291809de59fd14ae0134ac8a: Status 404 returned error can't find the container with id a2d2a75ff4b82f08c13fe2bc0b8ba9505b44b6a5291809de59fd14ae0134ac8a Jul 14 11:17:53.683372 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.683345 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296e9775_ef62_455d_9513_1fd1e4358764.slice/crio-b867e231d349ba4f6f72478a1a62877bed1907f36624412b3b4c8943e4a82150 WatchSource:0}: Error finding container b867e231d349ba4f6f72478a1a62877bed1907f36624412b3b4c8943e4a82150: Status 404 returned error can't find the container with id b867e231d349ba4f6f72478a1a62877bed1907f36624412b3b4c8943e4a82150 Jul 14 11:17:53.683745 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.683653 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" event={"ID":"deb191ef-3e9d-4ae6-afa1-84f41d02d051","Type":"ContainerStarted","Data":"7f2ad8e3df32030ad75655817262841feedb20974de75e3b72f67dd6d52723a4"} Jul 14 11:17:53.685062 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:53.685038 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hxzbm" event={"ID":"49806619-4c59-4b80-b24f-22162de9877b","Type":"ContainerStarted","Data":"a2d2a75ff4b82f08c13fe2bc0b8ba9505b44b6a5291809de59fd14ae0134ac8a"} Jul 14 11:17:53.705175 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.705151 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c57b79_de41_4891_a513_7102b9c8ffc3.slice/crio-0b8376fc19bb72de9f176c26bc8056026c8437073d9257bfe6f8bbd4e1a62572 WatchSource:0}: Error finding container 0b8376fc19bb72de9f176c26bc8056026c8437073d9257bfe6f8bbd4e1a62572: Status 404 returned error can't find the container with id 0b8376fc19bb72de9f176c26bc8056026c8437073d9257bfe6f8bbd4e1a62572 Jul 14 11:17:53.706024 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.705997 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442cf746_b413_40eb_906f_edac4e19044f.slice/crio-f8466fcf26ddc1ca376328762738a96b9dabb734ec152665a82f202b582dd5c6 WatchSource:0}: Error finding container f8466fcf26ddc1ca376328762738a96b9dabb734ec152665a82f202b582dd5c6: Status 404 returned error can't find the container with id f8466fcf26ddc1ca376328762738a96b9dabb734ec152665a82f202b582dd5c6 Jul 14 11:17:53.707373 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.706971 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511a3de3_f45f_4efe_899d_ec67102344ab.slice/crio-43c1c311642150b1cec8d2c3c0f315747e8138eef8ece7f14bb17c75ae960cd5 WatchSource:0}: Error finding container 43c1c311642150b1cec8d2c3c0f315747e8138eef8ece7f14bb17c75ae960cd5: Status 404 returned error can't find the container with id 43c1c311642150b1cec8d2c3c0f315747e8138eef8ece7f14bb17c75ae960cd5 Jul 14 11:17:53.709000 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:17:53.708357 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e44be89_e8f1_41ce_94c6_ee178019b118.slice/crio-f5026afee95c6ec56625696d9eb82be5dd94a6e96f16dbe8d26db5e322c20826 WatchSource:0}: Error finding container f5026afee95c6ec56625696d9eb82be5dd94a6e96f16dbe8d26db5e322c20826: Status 404 returned error can't find the container with id f5026afee95c6ec56625696d9eb82be5dd94a6e96f16dbe8d26db5e322c20826 Jul 14 11:17:54.285583 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.284734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:54.285583 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.284786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:54.285583 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.284968 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:54.285583 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.285033 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:17:56.285013835 +0000 UTC m=+6.140847991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:54.285583 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.285463 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:54.285583 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.285519 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:17:56.285501296 +0000 UTC m=+6.141335453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:54.385565 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.385523 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:54.385758 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.385740 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:17:54.385820 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.385766 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:17:54.385820 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.385779 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:54.385923 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.385841 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:17:56.385820596 +0000 UTC m=+6.241654763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:54.596865 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.596746 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:12:52 +0000 UTC" deadline="2028-02-07 22:40:02.332329554 +0000 UTC" Jul 14 11:17:54.596865 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.596812 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13763h22m7.735521803s" Jul 14 11:17:54.672892 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.672116 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:54.672892 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.672295 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:17:54.672892 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.672734 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:54.672892 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.672847 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:17:54.673708 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.673570 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:54.673708 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:54.673669 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:17:54.693532 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.693372 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t2jvz" event={"ID":"445082e1-8f80-4e41-8164-2a8ce6a18b10","Type":"ContainerStarted","Data":"3d716d6f2235c506c5450ee8b0eb8585e6ec75078ff89c7eed02b5d913c48cc5"} Jul 14 11:17:54.699409 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.699334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8ns2q" event={"ID":"511a3de3-f45f-4efe-899d-ec67102344ab","Type":"ContainerStarted","Data":"43c1c311642150b1cec8d2c3c0f315747e8138eef8ece7f14bb17c75ae960cd5"} Jul 14 11:17:54.709236 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.709203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" event={"ID":"525ade16321a29f4f96b69f63298cd87","Type":"ContainerStarted","Data":"fd31cba672e1f8648db7fdc75c63d22077e9094c22ddb2b44e1dd32366e2469a"} Jul 14 11:17:54.715084 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.714937 2568 generic.go:356] "Generic (PLEG): container finished" podID="bf7b9f377dc657d6e2281ba56c987b9b" containerID="90a0c1af1890eeb49dcaec1d6e23e1406fde5ceb7b7e979651e79f7aea3a32e7" exitCode=0 Jul 14 11:17:54.715084 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.715027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" event={"ID":"bf7b9f377dc657d6e2281ba56c987b9b","Type":"ContainerDied","Data":"90a0c1af1890eeb49dcaec1d6e23e1406fde5ceb7b7e979651e79f7aea3a32e7"} Jul 14 11:17:54.720600 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.720212 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerStarted","Data":"9485c771479e756854eb8d1d14d6548a776981643f6b262a2a671fa1672146da"} Jul 14 11:17:54.726995 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.726944 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-255.ec2.internal" podStartSLOduration=2.726922877 podStartE2EDuration="2.726922877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:17:54.726221917 +0000 UTC m=+4.582056093" watchObservedRunningTime="2026-07-14 11:17:54.726922877 +0000 UTC m=+4.582757052" Jul 14 11:17:54.733524 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.733494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9bq6b" event={"ID":"442cf746-b413-40eb-906f-edac4e19044f","Type":"ContainerStarted","Data":"f8466fcf26ddc1ca376328762738a96b9dabb734ec152665a82f202b582dd5c6"} Jul 14 11:17:54.740692 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.740528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"0b8376fc19bb72de9f176c26bc8056026c8437073d9257bfe6f8bbd4e1a62572"} Jul 14 11:17:54.742551 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.742523 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rlv7" event={"ID":"296e9775-ef62-455d-9513-1fd1e4358764","Type":"ContainerStarted","Data":"b867e231d349ba4f6f72478a1a62877bed1907f36624412b3b4c8943e4a82150"} Jul 14 11:17:54.752551 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:54.752524 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" event={"ID":"6e44be89-e8f1-41ce-94c6-ee178019b118","Type":"ContainerStarted","Data":"f5026afee95c6ec56625696d9eb82be5dd94a6e96f16dbe8d26db5e322c20826"} Jul 14 11:17:55.765359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:55.765317 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" event={"ID":"bf7b9f377dc657d6e2281ba56c987b9b","Type":"ContainerStarted","Data":"36e234f5b1bb6cdd4bd1ffb366e14a3cf080f98794c0b25be8aa36bcfcc6567b"} Jul 14 11:17:56.304036 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:56.303996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:56.304218 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:56.304049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:56.304218 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.304178 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:56.304398 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.304242 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:00.304220101 +0000 UTC m=+10.160054256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:17:56.304398 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.304323 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:56.304398 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.304357 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:00.304345866 +0000 UTC m=+10.160180015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:17:56.405227 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:56.405189 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:56.405451 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.405430 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:17:56.405512 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.405460 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:17:56.405512 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.405473 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:56.405617 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.405532 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:00.405513216 +0000 UTC m=+10.261347372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:17:56.674298 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:56.674208 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:56.674459 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.674356 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:17:56.674745 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:56.674725 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:56.674839 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.674819 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:17:56.674899 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:56.674881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:56.675004 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:56.674965 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:17:58.672460 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:58.672242 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:17:58.672460 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:58.672276 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:17:58.672460 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:58.672397 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:17:58.673125 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:58.672488 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:17:58.673125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:17:58.672545 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:17:58.673125 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:17:58.672619 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:00.343892 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:00.343853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:00.344373 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:00.343902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:00.344373 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.344033 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:00.344373 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.344041 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:00.344373 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.344103 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:08.344085271 +0000 UTC m=+18.199919422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:00.344373 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.344121 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:08.344112812 +0000 UTC m=+18.199946962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:00.445037 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:00.444995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:00.445214 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.445149 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:00.445214 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.445169 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:00.445214 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.445179 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:00.445411 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.445230 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:08.445213068 +0000 UTC m=+18.301047232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:00.674516 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:00.673381 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:00.674516 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:00.673381 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:00.674516 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.674131 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:00.674516 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.674502 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:00.674809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:00.674550 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:00.674809 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:00.674609 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:02.672274 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:02.672227 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:02.672274 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:02.672246 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:02.672731 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:02.672374 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:02.676716 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:02.673849 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:02.676716 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:02.673965 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:02.676716 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:02.675575 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:04.674844 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:04.674816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:04.675312 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:04.674816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:04.675312 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:04.674943 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:04.675312 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:04.674816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:04.675312 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:04.675027 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:04.675312 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:04.675103 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:06.672134 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:06.672099 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:06.672545 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:06.672100 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:06.672545 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:06.672236 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:06.672545 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:06.672249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:06.672545 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:06.672392 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:06.672545 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:06.672478 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:08.410876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:08.410835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:08.411372 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:08.410883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:08.411372 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.411010 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:08.411372 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.411071 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:24.4110509 +0000 UTC m=+34.266885053 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:08.411570 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.411489 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:08.411570 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.411547 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:24.411530963 +0000 UTC m=+34.267365114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:08.511537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:08.511502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:08.511726 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.511707 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:08.511806 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.511737 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:08.511806 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.511750 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:08.511898 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.511836 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:24.511814942 +0000 UTC m=+34.367649112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:08.672501 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:08.672428 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:08.672668 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.672563 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:08.672668 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:08.672441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:08.672668 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.672650 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:08.672668 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:08.672428 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:08.672859 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:08.672723 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:10.673045 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:10.672998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:10.673498 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:10.673117 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:10.673498 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:10.673137 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:10.673498 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:10.673191 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:10.673498 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:10.673240 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:10.673498 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:10.673380 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:12.672394 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:12.672353 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:12.672915 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:12.672357 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:12.672915 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:12.672499 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:12.672915 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:12.672567 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:12.672915 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:12.672356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:12.672915 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:12.672653 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:13.805392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.804985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hxzbm" event={"ID":"49806619-4c59-4b80-b24f-22162de9877b","Type":"ContainerStarted","Data":"03f2429402539d1fceea8da208aebb8a067b268ff287b86c22d6ceff65d1bf7e"} Jul 14 11:18:13.806538 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.806515 2568 generic.go:356] "Generic (PLEG): container finished" podID="eafa3614-4086-44e2-93e9-8839573a9397" containerID="e4c636fae3f5243a69c9ae17df3d088536288afb41a2cafd64f8f40ca63a4fca" exitCode=0 Jul 14 11:18:13.806655 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.806632 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerDied","Data":"e4c636fae3f5243a69c9ae17df3d088536288afb41a2cafd64f8f40ca63a4fca"} Jul 14 11:18:13.808339 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.808308 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9bq6b" event={"ID":"442cf746-b413-40eb-906f-edac4e19044f","Type":"ContainerStarted","Data":"3b5d14ed9ee7e14260928d8c078298e726e2ff9250a687acb2acdb72a3b667c8"} Jul 14 11:18:13.811832 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.811813 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:18:13.812424 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812401 2568 generic.go:356] "Generic (PLEG): container finished" podID="45c57b79-de41-4891-a513-7102b9c8ffc3" containerID="ad0dfbeb0f7ec22c09994905e26def3d786b51571d847c54d339f657aff6c5e3" exitCode=1 Jul 14 11:18:13.812516 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812466 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"5e59edd944ef62fe92c8e5236e0b07390688b75aa5475719d5a76d4931ebd6af"} Jul 14 11:18:13.812591 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812544 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"3157e05864cbcb3741f125f40e862208b59e321e596023d87c02b7d867cdc20b"} Jul 14 11:18:13.812637 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812592 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"fa69a2ce74d48d67b9f855971499edc4c68d0e5faf8fe6736cfcd17372770400"} Jul 14 11:18:13.812637 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812625 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"d232aa0158b875576469b8898f4fcf4d69e76d291eeec54023abb37325ec1c7b"} Jul 14 11:18:13.812729 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812658 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerDied","Data":"ad0dfbeb0f7ec22c09994905e26def3d786b51571d847c54d339f657aff6c5e3"} Jul 14 11:18:13.812729 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.812701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"e3ac4791603d6d7cc1a4af1d5e1664ee617b061c2a39703de965e5446e99aac2"} Jul 14 11:18:13.814900 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.814287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rlv7" event={"ID":"296e9775-ef62-455d-9513-1fd1e4358764","Type":"ContainerStarted","Data":"726593250dc8cb31df0a078d85f50b02979e25fa120a60bbb92c6141c27a68c3"} Jul 14 11:18:13.816240 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.816216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" event={"ID":"deb191ef-3e9d-4ae6-afa1-84f41d02d051","Type":"ContainerStarted","Data":"0dc859def49f0f9beb6be7dc999b18fb2991bb904d8388c6801a36533689eba3"} Jul 14 11:18:13.817806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.817784 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" event={"ID":"6e44be89-e8f1-41ce-94c6-ee178019b118","Type":"ContainerStarted","Data":"e91592c0ecac4b16dd1af70873cbe361b3d9d1cfd076b42f95d6973cc6b78cdf"} Jul 14 11:18:13.819232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.819208 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t2jvz" event={"ID":"445082e1-8f80-4e41-8164-2a8ce6a18b10","Type":"ContainerStarted","Data":"cf5a8b6c1ba0e67fc8258255aca3ff0adfd6bd5dcb4a3db91f9bf3ae4279c881"} Jul 14 11:18:13.823834 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.823796 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hxzbm" podStartSLOduration=4.394587934 podStartE2EDuration="23.823783885s" totalImagesPullingTime="19.429195951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:13.823486455 +0000 UTC m=+23.679320626" watchObservedRunningTime="2026-07-14 11:18:13.823783885 +0000 UTC m=+23.679618057" Jul 14 11:18:13.823938 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.823879 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-255.ec2.internal" podStartSLOduration=21.823873147 podStartE2EDuration="21.823873147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:17:55.783109339 +0000 UTC m=+5.638943514" watchObservedRunningTime="2026-07-14 11:18:13.823873147 +0000 UTC m=+23.679707319" Jul 14 11:18:13.865734 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.865687 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t2jvz" podStartSLOduration=3.463267704 podStartE2EDuration="22.865671832s" totalImagesPullingTime="19.402404128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:13.865490888 +0000 UTC m=+23.721325067" watchObservedRunningTime="2026-07-14 11:18:13.865671832 +0000 UTC m=+23.721506004" Jul 14 11:18:13.883809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.883729 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8rlv7" podStartSLOduration=3.474351692 podStartE2EDuration="22.883710275s" totalImagesPullingTime="19.409358583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:13.883010188 +0000 UTC m=+23.738844360" watchObservedRunningTime="2026-07-14 11:18:13.883710275 +0000 UTC m=+23.739544449" Jul 14 11:18:13.911017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.908767 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mr8rv" podStartSLOduration=3.510388314 podStartE2EDuration="22.9087461s" totalImagesPullingTime="19.398357786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:13.908468496 +0000 UTC m=+23.764302669" watchObservedRunningTime="2026-07-14 11:18:13.9087461 +0000 UTC m=+23.764580264" Jul 14 11:18:13.932905 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:13.932844 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9bq6b" podStartSLOduration=3.469744933 podStartE2EDuration="22.932824611s" totalImagesPullingTime="19.463079678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:13.932497878 +0000 UTC m=+23.788332051" watchObservedRunningTime="2026-07-14 11:18:13.932824611 +0000 UTC m=+23.788658784" Jul 14 11:18:14.672694 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.672665 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:14.672939 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.672677 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:14.672939 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:14.672791 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:14.672939 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:14.672886 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:14.673145 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.672949 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:14.673145 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:14.673050 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:14.789566 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.789534 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 11:18:14.823946 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.823920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" event={"ID":"deb191ef-3e9d-4ae6-afa1-84f41d02d051","Type":"ContainerStarted","Data":"1530fb24437b6c135dc5c4844c7bbfc43ff2b2408fb5564b770c6500d9ba375f"} Jul 14 11:18:14.825849 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.825822 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8ns2q" event={"ID":"511a3de3-f45f-4efe-899d-ec67102344ab","Type":"ContainerStarted","Data":"19d5ca187a2a614a3696192b1a0c6636568aabf6be700b21f80a5045d69fbd91"} Jul 14 11:18:14.843390 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:14.843352 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8ns2q" podStartSLOduration=4.430477685 podStartE2EDuration="23.843337944s" totalImagesPullingTime="19.412860259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:14.842755775 +0000 UTC m=+24.698589950" watchObservedRunningTime="2026-07-14 11:18:14.843337944 +0000 UTC m=+24.699172115" Jul 14 11:18:15.635918 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:15.635626 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T11:18:14.789558259Z","UUID":"5bab9cd6-6163-40b4-9cab-09eadc997f3c","Handler":null,"Name":"","Endpoint":""} Jul 14 11:18:15.637439 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:15.637401 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 11:18:15.637567 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:15.637447 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 11:18:16.649927 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.649900 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:18:16.650608 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.650594 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:18:16.672775 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.672749 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:16.672775 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.672761 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:16.672930 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.672749 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:16.672930 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:16.672859 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:16.672930 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:16.672913 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:16.673079 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:16.673012 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:16.834680 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.834647 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:18:16.835093 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.835064 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"c778cfabdaf6640eeb754ae84968c91ce47c1ceee57073e049925368333b47d9"} Jul 14 11:18:16.837488 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.837460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" event={"ID":"deb191ef-3e9d-4ae6-afa1-84f41d02d051","Type":"ContainerStarted","Data":"01f8251c76b878abfd07dd10ca7ef14d523685e92fbb05cc0367f1e6d0dfbb9f"} Jul 14 11:18:16.857105 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:16.857061 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltm2f" podStartSLOduration=4.762522058 podStartE2EDuration="26.85704686s" totalImagesPullingTime="22.094524802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:16.856990192 +0000 UTC m=+26.712824365" watchObservedRunningTime="2026-07-14 11:18:16.85704686 +0000 UTC m=+26.712881032" Jul 14 11:18:18.672245 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.672039 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:18.673132 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.672055 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:18.673132 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.672112 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:18.673132 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:18.672349 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:18.673132 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:18.672455 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:18.673132 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:18.672538 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:18.843606 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.843529 2568 generic.go:356] "Generic (PLEG): container finished" podID="eafa3614-4086-44e2-93e9-8839573a9397" containerID="de0881084b5a0db103f617c0655fd07e0278b7591ac8fbf01995fe2377c4258f" exitCode=0 Jul 14 11:18:18.843750 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.843608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerDied","Data":"de0881084b5a0db103f617c0655fd07e0278b7591ac8fbf01995fe2377c4258f"} Jul 14 11:18:18.847173 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.847153 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:18:18.847552 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.847529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"2fd3defca06305f39aae135d3306b85c0e2616ef547750fd4e0be775d5119631"} Jul 14 11:18:18.847828 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.847811 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:18:18.847924 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.847900 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:18:18.848059 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.848042 2568 scope.go:117] "RemoveContainer" containerID="ad0dfbeb0f7ec22c09994905e26def3d786b51571d847c54d339f657aff6c5e3" Jul 14 11:18:18.865186 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:18.865166 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:18:19.853722 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:19.853699 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:18:19.854195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:19.854012 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" event={"ID":"45c57b79-de41-4891-a513-7102b9c8ffc3","Type":"ContainerStarted","Data":"4bbd76881f2886fda13d21bc69d57e724b58a091135b47d24c9aa7cb1269f43c"} Jul 14 11:18:19.854390 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:19.854367 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:18:19.869974 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:19.869951 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:18:19.906175 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:19.906128 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" podStartSLOduration=9.440361366 podStartE2EDuration="28.906115059s" totalImagesPullingTime="19.465753693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:19.905659003 +0000 UTC m=+29.761493176" watchObservedRunningTime="2026-07-14 11:18:19.906115059 +0000 UTC m=+29.761949231" Jul 14 11:18:20.632971 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.632942 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tffnh"] Jul 14 11:18:20.633212 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.633077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:20.633212 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:20.633159 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:20.635988 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.635966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8dclj"] Jul 14 11:18:20.636092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.636061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:20.636155 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:20.636141 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:20.645140 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.645112 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2qzv4"] Jul 14 11:18:20.645225 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.645205 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:20.645339 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:20.645318 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:20.858626 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.858449 2568 generic.go:356] "Generic (PLEG): container finished" podID="eafa3614-4086-44e2-93e9-8839573a9397" containerID="9299bb863d497ba64445d4e82b0bad9391b87317c22a346be0990c922a5893d2" exitCode=0 Jul 14 11:18:20.859058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.858529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerDied","Data":"9299bb863d497ba64445d4e82b0bad9391b87317c22a346be0990c922a5893d2"} Jul 14 11:18:20.967930 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.967839 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:18:20.968093 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.967979 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 11:18:20.968519 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:20.968497 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8rlv7" Jul 14 11:18:21.672396 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:21.672369 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:21.672553 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:21.672475 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:22.672022 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:22.671988 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:22.672660 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:22.671991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:22.672660 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:22.672119 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:22.672660 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:22.672182 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:22.865327 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:22.865224 2568 generic.go:356] "Generic (PLEG): container finished" podID="eafa3614-4086-44e2-93e9-8839573a9397" containerID="8870b3b315ebff6c9ae3f58a15451c6a0c2744e9bc354763d17cc0c68e23575a" exitCode=0 Jul 14 11:18:22.865327 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:22.865304 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerDied","Data":"8870b3b315ebff6c9ae3f58a15451c6a0c2744e9bc354763d17cc0c68e23575a"} Jul 14 11:18:23.672037 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:23.671975 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:23.672603 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:23.672104 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:24.443485 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:24.443454 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:24.443674 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:24.443498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:24.443674 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.443611 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:24.443674 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.443626 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:24.443674 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.443667 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret podName:7e1eb7e2-1e05-4af6-93f1-894e406169c9 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.443655485 +0000 UTC m=+66.299489634 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret") pod "global-pull-secret-syncer-2qzv4" (UID: "7e1eb7e2-1e05-4af6-93f1-894e406169c9") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:24.443893 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.443681 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.443675026 +0000 UTC m=+66.299509176 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:24.544654 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:24.544619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:24.544806 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.544770 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:24.544806 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.544788 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:24.544806 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.544798 2568 projected.go:196] Error preparing data for projected volume kube-api-access-2bp2d for pod openshift-network-diagnostics/network-check-target-tffnh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:24.544955 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.544856 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d podName:c70ba287-c702-4f55-bc66-ed761b25607c nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.544836412 +0000 UTC m=+66.400670589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2bp2d" (UniqueName: "kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d") pod "network-check-target-tffnh" (UID: "c70ba287-c702-4f55-bc66-ed761b25607c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:24.672138 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:24.672107 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:24.672536 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.672237 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:24.672536 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:24.672298 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:24.672660 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:24.672384 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:25.671793 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:25.671752 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:25.671987 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:25.671894 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dclj" podUID="ecc55306-170e-463f-b3e5-263e3201fc1c" Jul 14 11:18:26.672446 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:26.672417 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:26.672759 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:26.672549 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2qzv4" podUID="7e1eb7e2-1e05-4af6-93f1-894e406169c9" Jul 14 11:18:26.672759 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:26.672609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:26.672759 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:26.672722 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tffnh" podUID="c70ba287-c702-4f55-bc66-ed761b25607c" Jul 14 11:18:26.962214 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:26.962193 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-255.ec2.internal" event="NodeReady" Jul 14 11:18:26.962333 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:26.962321 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 11:18:27.005965 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.005110 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q"] Jul 14 11:18:27.007922 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.007898 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj"] Jul 14 11:18:27.008058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.008024 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.009778 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.009757 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c"] Jul 14 11:18:27.009887 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.009872 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.011657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.011637 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-678db546b5-v268x"] Jul 14 11:18:27.011818 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.011742 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.012564 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.012152 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 11:18:27.012564 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.012184 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 11:18:27.012564 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.012233 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 11:18:27.012785 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.012614 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 11:18:27.012785 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.012653 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 11:18:27.013170 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.013152 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.013784 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.013767 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6qx2d\"" Jul 14 11:18:27.014462 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.014444 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 14 11:18:27.016496 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.016477 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 14 11:18:27.017696 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.017220 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 11:18:27.017696 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.017389 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rhbkm\"" Jul 14 11:18:27.019704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.018316 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 11:18:27.019704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.018722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 11:18:27.019704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.018756 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 14 11:18:27.019704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.018865 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 14 11:18:27.023655 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.023628 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj"] Jul 14 11:18:27.024818 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.024795 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q"] Jul 14 11:18:27.026001 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.025983 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 11:18:27.029373 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.029354 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mq2gr"] Jul 14 11:18:27.031675 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.031651 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t9jdh"] Jul 14 11:18:27.032515 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.032446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.035075 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.035033 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c"] Jul 14 11:18:27.035359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.035340 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 11:18:27.035681 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.035657 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgqwk\"" Jul 14 11:18:27.035895 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.035878 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-678db546b5-v268x"] Jul 14 11:18:27.035978 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.035941 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.036645 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.036625 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 11:18:27.039541 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.039522 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 11:18:27.039705 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.039686 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 11:18:27.039919 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.039901 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 11:18:27.040294 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.040280 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9c655\"" Jul 14 11:18:27.045639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.045619 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9jdh"] Jul 14 11:18:27.050429 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.050407 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mq2gr"] Jul 14 11:18:27.059815 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.059790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw295\" (UniqueName: \"kubernetes.io/projected/603dbaa9-96bd-442e-b074-9f65acb52bae-kube-api-access-pw295\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.059993 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.059830 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648326eb-7168-4bdb-a3e0-e7b510948491-cert\") pod \"ingress-canary-t9jdh\" (UID: \"648326eb-7168-4bdb-a3e0-e7b510948491\") " pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.059993 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.059856 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.060527 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96vn8\" (UniqueName: \"kubernetes.io/projected/53beda37-da6d-4cff-88c6-8341416a83de-kube-api-access-96vn8\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.060634 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060549 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8r6d\" (UniqueName: \"kubernetes.io/projected/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-kube-api-access-n8r6d\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.060634 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53beda37-da6d-4cff-88c6-8341416a83de-config-volume\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.060769 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060647 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.060769 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvsg5\" (UniqueName: \"kubernetes.io/projected/648326eb-7168-4bdb-a3e0-e7b510948491-kube-api-access-kvsg5\") pod \"ingress-canary-t9jdh\" (UID: \"648326eb-7168-4bdb-a3e0-e7b510948491\") " pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.060769 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060727 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4htp\" (UniqueName: \"kubernetes.io/projected/8a016414-cf9c-4420-be91-0a91de5abed5-kube-api-access-l4htp\") pod \"managed-serviceaccount-addon-agent-599b4cf84d-p827q\" (UID: \"8a016414-cf9c-4420-be91-0a91de5abed5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.060876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-image-registry-private-configuration\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.060876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060798 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-trusted-ca\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.060876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-installation-pull-secrets\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.060876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5b65\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-kube-api-access-c5b65\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.061005 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060900 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-bound-sa-token\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.061005 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed757a72-785a-4619-b72d-aa643cf0ae32-ca-trust-extracted\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.061005 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.060974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-certificates\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.061125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061001 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/603dbaa9-96bd-442e-b074-9f65acb52bae-tmp\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.061125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061037 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/603dbaa9-96bd-442e-b074-9f65acb52bae-klusterlet-config\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.061125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-hub\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.061224 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061140 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53beda37-da6d-4cff-88c6-8341416a83de-tmp-dir\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.061224 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061171 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-ca\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.061224 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.061381 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.061381 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061342 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.061459 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.061392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a016414-cf9c-4420-be91-0a91de5abed5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-599b4cf84d-p827q\" (UID: \"8a016414-cf9c-4420-be91-0a91de5abed5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.162302 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648326eb-7168-4bdb-a3e0-e7b510948491-cert\") pod \"ingress-canary-t9jdh\" (UID: \"648326eb-7168-4bdb-a3e0-e7b510948491\") " pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.162302 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162304 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162326 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96vn8\" (UniqueName: \"kubernetes.io/projected/53beda37-da6d-4cff-88c6-8341416a83de-kube-api-access-96vn8\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162342 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8r6d\" (UniqueName: \"kubernetes.io/projected/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-kube-api-access-n8r6d\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53beda37-da6d-4cff-88c6-8341416a83de-config-volume\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162408 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvsg5\" (UniqueName: \"kubernetes.io/projected/648326eb-7168-4bdb-a3e0-e7b510948491-kube-api-access-kvsg5\") pod \"ingress-canary-t9jdh\" (UID: \"648326eb-7168-4bdb-a3e0-e7b510948491\") " pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4htp\" (UniqueName: \"kubernetes.io/projected/8a016414-cf9c-4420-be91-0a91de5abed5-kube-api-access-l4htp\") pod \"managed-serviceaccount-addon-agent-599b4cf84d-p827q\" (UID: \"8a016414-cf9c-4420-be91-0a91de5abed5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.162546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162473 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-image-registry-private-configuration\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.162557 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.162628 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:18:27.662605328 +0000 UTC m=+37.518439481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-trusted-ca\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-installation-pull-secrets\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5b65\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-kube-api-access-c5b65\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-bound-sa-token\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162821 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed757a72-785a-4619-b72d-aa643cf0ae32-ca-trust-extracted\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.162886 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-certificates\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/603dbaa9-96bd-442e-b074-9f65acb52bae-tmp\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/603dbaa9-96bd-442e-b074-9f65acb52bae-klusterlet-config\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.162982 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-hub\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53beda37-da6d-4cff-88c6-8341416a83de-tmp-dir\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163028 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53beda37-da6d-4cff-88c6-8341416a83de-config-volume\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-ca\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a016414-cf9c-4420-be91-0a91de5abed5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-599b4cf84d-p827q\" (UID: \"8a016414-cf9c-4420-be91-0a91de5abed5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163193 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw295\" (UniqueName: \"kubernetes.io/projected/603dbaa9-96bd-442e-b074-9f65acb52bae-kube-api-access-pw295\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.163288 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163227 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/603dbaa9-96bd-442e-b074-9f65acb52bae-tmp\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.163870 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163362 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed757a72-785a-4619-b72d-aa643cf0ae32-ca-trust-extracted\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.163870 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163473 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53beda37-da6d-4cff-88c6-8341416a83de-tmp-dir\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.163870 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.163853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-certificates\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.164003 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.163935 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:27.164003 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.163946 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:27.164085 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.163987 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:27.663971553 +0000 UTC m=+37.519805728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:27.165191 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.164867 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-trusted-ca\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.165191 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.164954 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.168156 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168135 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.168235 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648326eb-7168-4bdb-a3e0-e7b510948491-cert\") pod \"ingress-canary-t9jdh\" (UID: \"648326eb-7168-4bdb-a3e0-e7b510948491\") " pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.168318 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168247 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-hub\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.168379 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168341 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.168379 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168352 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/603dbaa9-96bd-442e-b074-9f65acb52bae-klusterlet-config\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.168379 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168358 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a016414-cf9c-4420-be91-0a91de5abed5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-599b4cf84d-p827q\" (UID: \"8a016414-cf9c-4420-be91-0a91de5abed5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.168597 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168578 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-installation-pull-secrets\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.168683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.168667 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-ca\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.169098 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.169081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-image-registry-private-configuration\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.171484 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.171462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96vn8\" (UniqueName: \"kubernetes.io/projected/53beda37-da6d-4cff-88c6-8341416a83de-kube-api-access-96vn8\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.172310 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.172294 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvsg5\" (UniqueName: \"kubernetes.io/projected/648326eb-7168-4bdb-a3e0-e7b510948491-kube-api-access-kvsg5\") pod \"ingress-canary-t9jdh\" (UID: \"648326eb-7168-4bdb-a3e0-e7b510948491\") " pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.175944 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.175913 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-bound-sa-token\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.176657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.176629 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5b65\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-kube-api-access-c5b65\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.176948 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.176929 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw295\" (UniqueName: \"kubernetes.io/projected/603dbaa9-96bd-442e-b074-9f65acb52bae-kube-api-access-pw295\") pod \"klusterlet-addon-workmgr-7976fcc5ff-fgxtj\" (UID: \"603dbaa9-96bd-442e-b074-9f65acb52bae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.177042 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.176934 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8r6d\" (UniqueName: \"kubernetes.io/projected/5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b-kube-api-access-n8r6d\") pod \"cluster-proxy-proxy-agent-775b664c7-bml4c\" (UID: \"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.177349 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.177332 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4htp\" (UniqueName: \"kubernetes.io/projected/8a016414-cf9c-4420-be91-0a91de5abed5-kube-api-access-l4htp\") pod \"managed-serviceaccount-addon-agent-599b4cf84d-p827q\" (UID: \"8a016414-cf9c-4420-be91-0a91de5abed5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.319301 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.319243 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:27.334137 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.334112 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" Jul 14 11:18:27.342752 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.342731 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:27.348682 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.348659 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:18:27.425809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.425426 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9jdh" Jul 14 11:18:27.448211 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.447959 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:27.538939 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.538906 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q"] Jul 14 11:18:27.542916 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:27.542891 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a016414_cf9c_4420_be91_0a91de5abed5.slice/crio-49ebbe2d99d39110fd6b10fa6c98cef8ab85b8c24d459ca3549464417ea3a78f WatchSource:0}: Error finding container 49ebbe2d99d39110fd6b10fa6c98cef8ab85b8c24d459ca3549464417ea3a78f: Status 404 returned error can't find the container with id 49ebbe2d99d39110fd6b10fa6c98cef8ab85b8c24d459ca3549464417ea3a78f Jul 14 11:18:27.548328 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.548307 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj"] Jul 14 11:18:27.551577 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:27.551545 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603dbaa9_96bd_442e_b074_9f65acb52bae.slice/crio-1d7341f1d0ec05115fa9d5781cce04ebe0f9c33fe1e82f8e9e4f866768a43bc9 WatchSource:0}: Error finding container 1d7341f1d0ec05115fa9d5781cce04ebe0f9c33fe1e82f8e9e4f866768a43bc9: Status 404 returned error can't find the container with id 1d7341f1d0ec05115fa9d5781cce04ebe0f9c33fe1e82f8e9e4f866768a43bc9 Jul 14 11:18:27.564880 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.564856 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c"] Jul 14 11:18:27.570751 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:27.570730 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ec5b0ad_926a_4d10_8d21_88e0c6bcee4b.slice/crio-2d92eaac0d196b5ab3adc1b16d9b9d2a5bfcbc58e11ac23d4cb522b327506034 WatchSource:0}: Error finding container 2d92eaac0d196b5ab3adc1b16d9b9d2a5bfcbc58e11ac23d4cb522b327506034: Status 404 returned error can't find the container with id 2d92eaac0d196b5ab3adc1b16d9b9d2a5bfcbc58e11ac23d4cb522b327506034 Jul 14 11:18:27.573799 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.573777 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:27.589470 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.589448 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9jdh"] Jul 14 11:18:27.592064 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:27.592042 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648326eb_7168_4bdb_a3e0_e7b510948491.slice/crio-0230da861822be2cb5c1ae102370d42c66fd25a47390374a719f2c5f9c56e9bb WatchSource:0}: Error finding container 0230da861822be2cb5c1ae102370d42c66fd25a47390374a719f2c5f9c56e9bb: Status 404 returned error can't find the container with id 0230da861822be2cb5c1ae102370d42c66fd25a47390374a719f2c5f9c56e9bb Jul 14 11:18:27.668678 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.668645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:27.668840 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.668699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:27.668840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.668798 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:27.668840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.668811 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:27.668840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.668810 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:27.669054 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.668870 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:28.668851309 +0000 UTC m=+38.524685461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:27.669054 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:27.668887 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:18:28.66887903 +0000 UTC m=+38.524713180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:27.672725 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.672702 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:27.675609 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.675588 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 11:18:27.675729 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.675719 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bm9fn\"" Jul 14 11:18:27.706773 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.706751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:27.808440 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.808415 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:27.879043 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.879009 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" event={"ID":"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b","Type":"ContainerStarted","Data":"2d92eaac0d196b5ab3adc1b16d9b9d2a5bfcbc58e11ac23d4cb522b327506034"} Jul 14 11:18:27.881809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.881785 2568 generic.go:356] "Generic (PLEG): container finished" podID="eafa3614-4086-44e2-93e9-8839573a9397" containerID="69456e6a999d523429953efa288b21b9e1f4dee642eb0cfd76758420e87d4a32" exitCode=0 Jul 14 11:18:27.881947 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.881855 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerDied","Data":"69456e6a999d523429953efa288b21b9e1f4dee642eb0cfd76758420e87d4a32"} Jul 14 11:18:27.883426 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.883068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" event={"ID":"8a016414-cf9c-4420-be91-0a91de5abed5","Type":"ContainerStarted","Data":"49ebbe2d99d39110fd6b10fa6c98cef8ab85b8c24d459ca3549464417ea3a78f"} Jul 14 11:18:27.884168 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.884144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9jdh" event={"ID":"648326eb-7168-4bdb-a3e0-e7b510948491","Type":"ContainerStarted","Data":"0230da861822be2cb5c1ae102370d42c66fd25a47390374a719f2c5f9c56e9bb"} Jul 14 11:18:27.885356 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:27.885337 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" event={"ID":"603dbaa9-96bd-442e-b074-9f65acb52bae","Type":"ContainerStarted","Data":"1d7341f1d0ec05115fa9d5781cce04ebe0f9c33fe1e82f8e9e4f866768a43bc9"} Jul 14 11:18:28.512651 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.512366 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:28.586892 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.586844 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:28.673318 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.672419 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:28.673318 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.673202 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:28.676285 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.676240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:28.676403 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.676344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:28.676484 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:28.676467 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:28.676544 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:28.676489 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:28.676544 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:28.676542 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:30.676525 +0000 UTC m=+40.532359153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:28.676654 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:28.676609 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:28.676654 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:28.676642 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:18:30.676631056 +0000 UTC m=+40.532465210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:28.677402 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.677381 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 11:18:28.681652 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.681626 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.681866 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.681848 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.682084 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.682069 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q2rq4\"" Jul 14 11:18:28.903405 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.903009 2568 generic.go:356] "Generic (PLEG): container finished" podID="eafa3614-4086-44e2-93e9-8839573a9397" containerID="cbb8b53693caba8aa0b9286ff65acfa3d95c158f0a2e7cb5eeaabe3c35725af6" exitCode=0 Jul 14 11:18:28.903405 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.903084 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerDied","Data":"cbb8b53693caba8aa0b9286ff65acfa3d95c158f0a2e7cb5eeaabe3c35725af6"} Jul 14 11:18:28.907323 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.906936 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:28.983318 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:28.983248 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:29.916740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:29.916701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-766fl" event={"ID":"eafa3614-4086-44e2-93e9-8839573a9397","Type":"ContainerStarted","Data":"e78225b476f6da77ef1b777089a07380a110c3c6b240fb93e6b9b879735fdd53"} Jul 14 11:18:29.943006 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:29.942946 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-766fl" podStartSLOduration=8.560713159 podStartE2EDuration="38.942924926s" totalImagesPullingTime="30.382211767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:29.94238459 +0000 UTC m=+39.798218764" watchObservedRunningTime="2026-07-14 11:18:29.942924926 +0000 UTC m=+39.798759097" Jul 14 11:18:30.069587 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:30.069552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:30.165874 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:30.165662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:30.695273 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:30.695234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:30.695425 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:30.695350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:30.695500 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:30.695487 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:30.695570 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:30.695556 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:18:34.695537219 +0000 UTC m=+44.551371375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:30.696219 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:30.696199 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:30.696219 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:30.696220 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:30.696371 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:30.696282 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:34.696249099 +0000 UTC m=+44.552083258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:31.254280 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:31.254230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:31.355750 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:31.355710 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:31.455659 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:31.455622 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:34.728309 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:34.728267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:34.728772 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:34.728371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:34.728772 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:34.728448 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:34.728772 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:34.728466 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:34.728772 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:34.728474 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:34.728772 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:34.728524 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:18:42.728508308 +0000 UTC m=+52.584342463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:34.728772 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:34.728541 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:42.728533963 +0000 UTC m=+52.584368112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:35.934948 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.934886 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9jdh" event={"ID":"648326eb-7168-4bdb-a3e0-e7b510948491","Type":"ContainerStarted","Data":"cc803c92c5a91452ce9e47a3803ffd95527d4794c85f4f7d9984712191c0e7ff"} Jul 14 11:18:35.936376 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.936351 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" event={"ID":"603dbaa9-96bd-442e-b074-9f65acb52bae","Type":"ContainerStarted","Data":"1ce5270f6c8b7b78b7bb526969b156fe478e0b30e547aa75fd1daf9551af4466"} Jul 14 11:18:35.938792 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.938764 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" event={"ID":"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b","Type":"ContainerStarted","Data":"07158cc6fdc9288204ace3dc63cfdd29d3164b8b010ddddf2f9c79ece6d80279"} Jul 14 11:18:35.938893 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.938804 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" event={"ID":"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b","Type":"ContainerStarted","Data":"7e885f907879da5945e0d5b1024a523a893fd9697462c1060980d2a1b27312b0"} Jul 14 11:18:35.940075 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.940056 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" event={"ID":"8a016414-cf9c-4420-be91-0a91de5abed5","Type":"ContainerStarted","Data":"9f4b689f9272919a3dc5b429fb9998705287f6362d1da6458a5823d0b9cbd022"} Jul 14 11:18:35.959879 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.957271 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t9jdh" podStartSLOduration=0.907878826 podStartE2EDuration="8.957230861s" totalImagesPullingTime="8.049352035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:35.956780685 +0000 UTC m=+45.812614857" watchObservedRunningTime="2026-07-14 11:18:35.957230861 +0000 UTC m=+45.813065035" Jul 14 11:18:35.974204 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.974159 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599b4cf84d-p827q" podStartSLOduration=28.876223802 podStartE2EDuration="36.974143954s" totalImagesPullingTime="8.097920152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:35.973378013 +0000 UTC m=+45.829212185" watchObservedRunningTime="2026-07-14 11:18:35.974143954 +0000 UTC m=+45.829978126" Jul 14 11:18:35.993312 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:35.993247 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" podStartSLOduration=28.904163771 podStartE2EDuration="36.99322708s" totalImagesPullingTime="8.089063309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:35.992453665 +0000 UTC m=+45.848287836" watchObservedRunningTime="2026-07-14 11:18:35.99322708 +0000 UTC m=+45.849061254" Jul 14 11:18:36.944980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:36.944941 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" event={"ID":"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b","Type":"ContainerStarted","Data":"8fe01f04887d2a753d3bc5ce2250d555d4a0a4409c4eae4e9ae93d8d86da7cf4"} Jul 14 11:18:36.945544 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:36.945527 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:36.947055 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:36.947033 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7976fcc5ff-fgxtj" Jul 14 11:18:36.965222 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:36.965180 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" podStartSLOduration=29.877565496 podStartE2EDuration="37.965165639s" totalImagesPullingTime="8.087600143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:36.963954229 +0000 UTC m=+46.819788402" watchObservedRunningTime="2026-07-14 11:18:36.965165639 +0000 UTC m=+46.820999810" Jul 14 11:18:42.788493 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:42.788453 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:42.788874 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:42.788530 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:42.788874 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:42.788608 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:42.788874 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:42.788606 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:42.788874 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:42.788628 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:42.788874 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:42.788659 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:18:58.788645744 +0000 UTC m=+68.644479893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:42.788874 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:42.788676 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:58.788663982 +0000 UTC m=+68.644498135 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:48.872284 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.872236 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c"] Jul 14 11:18:48.874900 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.874878 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:48.875866 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.875845 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-794fd9f96d-dt6lg"] Jul 14 11:18:48.877324 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.877308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:48.877831 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.877814 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5jhwr\"" Jul 14 11:18:48.879196 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.879179 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 11:18:48.880653 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880633 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 11:18:48.880743 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880652 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 14 11:18:48.880743 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880631 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 14 11:18:48.880860 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880843 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 14 11:18:48.880917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880881 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 14 11:18:48.880917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880902 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 11:18:48.881020 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.880908 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 14 11:18:48.881095 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.881017 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mxzt8\"" Jul 14 11:18:48.881095 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.881025 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 11:18:48.881095 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.881074 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 14 11:18:48.886687 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.886666 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c"] Jul 14 11:18:48.893286 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.893249 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-794fd9f96d-dt6lg"] Jul 14 11:18:48.932417 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:48.932417 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932419 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8dcc\" (UniqueName: \"kubernetes.io/projected/00628d94-4a03-456d-8c41-0ab813236ad1-kube-api-access-n8dcc\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:48.932715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932441 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:48.932715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgqd9\" (UniqueName: \"kubernetes.io/projected/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-kube-api-access-tgqd9\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:48.932715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:48.932715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-stats-auth\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:48.932715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932593 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-default-certificate\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:48.932715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.932695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:48.980605 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.980582 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp"] Jul 14 11:18:48.982616 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.982598 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" Jul 14 11:18:48.985120 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.985095 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wl2m9"] Jul 14 11:18:48.986137 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.986115 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:48.986281 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.986120 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-td6qh\"" Jul 14 11:18:48.986538 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.986521 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:48.987117 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.987103 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-l8hd4"] Jul 14 11:18:48.987270 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.987232 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:48.988960 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.988944 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw"] Jul 14 11:18:48.989099 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.989085 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:48.992767 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.992741 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tqkvt\"" Jul 14 11:18:48.992933 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.992908 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 11:18:48.993065 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.992994 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:48.993065 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.993009 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 14 11:18:48.993224 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.993206 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 11:18:48.993507 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.993365 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:48.993748 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.993610 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 11:18:48.994611 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.994468 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 11:18:48.994734 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.994714 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 14 11:18:48.995023 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.995008 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bx5tw\"" Jul 14 11:18:48.995135 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:48.995107 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.001799 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.001777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2xvvv\"" Jul 14 11:18:49.001933 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.001900 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:49.002365 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.002345 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 11:18:49.002848 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.002829 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:49.003389 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.003371 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 14 11:18:49.003588 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.003004 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 11:18:49.003775 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.003658 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp"] Jul 14 11:18:49.006401 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.006381 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-l8hd4"] Jul 14 11:18:49.007555 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.007533 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wl2m9"] Jul 14 11:18:49.022684 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.022663 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw"] Jul 14 11:18:49.033632 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:49.033722 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgqd9\" (UniqueName: \"kubernetes.io/projected/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-kube-api-access-tgqd9\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:49.033722 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.033843 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.033778 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 11:18:49.033843 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xpqs\" (UniqueName: \"kubernetes.io/projected/090cb996-2729-46af-adac-941ef69bc0f8-kube-api-access-7xpqs\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.033843 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.033831 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:49.533813453 +0000 UTC m=+59.389647603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : secret "router-metrics-certs-default" not found Jul 14 11:18:49.034001 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ca8083f-bc69-461c-80e3-d5abc98eb611-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.034001 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-stats-auth\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.034001 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-default-certificate\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.034001 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.033989 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/090cb996-2729-46af-adac-941ef69bc0f8-config\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.034209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gbm5\" (UniqueName: \"kubernetes.io/projected/8ca8083f-bc69-461c-80e3-d5abc98eb611-kube-api-access-2gbm5\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.034209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8ca8083f-bc69-461c-80e3-d5abc98eb611-snapshots\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.034209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034083 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.034209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ca8083f-bc69-461c-80e3-d5abc98eb611-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.034209 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.034189 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:49.534174687 +0000 UTC m=+59.390008853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : configmap references non-existent config key: service-ca.crt Jul 14 11:18:49.034209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034235 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ca8083f-bc69-461c-80e3-d5abc98eb611-tmp\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034272 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/090cb996-2729-46af-adac-941ef69bc0f8-trusted-ca\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq82d\" (UniqueName: \"kubernetes.io/projected/ac55edc5-a5d4-4283-8b76-00d0fd56ec8e-kube-api-access-xq82d\") pod \"volume-data-source-validator-78495597f6-bdtbp\" (UID: \"ac55edc5-a5d4-4283-8b76-00d0fd56ec8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034316 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvbv\" (UniqueName: \"kubernetes.io/projected/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-kube-api-access-wjvbv\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034380 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8ca8083f-bc69-461c-80e3-d5abc98eb611-serving-cert\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034413 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/090cb996-2729-46af-adac-941ef69bc0f8-serving-cert\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:49.034517 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.034458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8dcc\" (UniqueName: \"kubernetes.io/projected/00628d94-4a03-456d-8c41-0ab813236ad1-kube-api-access-n8dcc\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.034852 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.034535 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:49.034852 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.034593 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls podName:17070e67-dd32-4d46-a9c1-7e2d634e2ccd nodeName:}" failed. No retries permitted until 2026-07-14 11:18:49.53457835 +0000 UTC m=+59.390412499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-rjc8c" (UID: "17070e67-dd32-4d46-a9c1-7e2d634e2ccd") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:49.037619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.037593 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-default-certificate\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.037703 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.037665 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-stats-auth\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.047576 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.047553 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgqd9\" (UniqueName: \"kubernetes.io/projected/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-kube-api-access-tgqd9\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:49.047652 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.047633 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8dcc\" (UniqueName: \"kubernetes.io/projected/00628d94-4a03-456d-8c41-0ab813236ad1-kube-api-access-n8dcc\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.134885 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.134798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/090cb996-2729-46af-adac-941ef69bc0f8-config\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.134885 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.134836 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gbm5\" (UniqueName: \"kubernetes.io/projected/8ca8083f-bc69-461c-80e3-d5abc98eb611-kube-api-access-2gbm5\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.134885 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.134861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8ca8083f-bc69-461c-80e3-d5abc98eb611-snapshots\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ca8083f-bc69-461c-80e3-d5abc98eb611-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.135190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ca8083f-bc69-461c-80e3-d5abc98eb611-tmp\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135152 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/090cb996-2729-46af-adac-941ef69bc0f8-trusted-ca\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.135190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq82d\" (UniqueName: \"kubernetes.io/projected/ac55edc5-a5d4-4283-8b76-00d0fd56ec8e-kube-api-access-xq82d\") pod \"volume-data-source-validator-78495597f6-bdtbp\" (UID: \"ac55edc5-a5d4-4283-8b76-00d0fd56ec8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" Jul 14 11:18:49.135190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135190 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvbv\" (UniqueName: \"kubernetes.io/projected/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-kube-api-access-wjvbv\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.135484 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.135241 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 11:18:49.135484 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8ca8083f-bc69-461c-80e3-d5abc98eb611-serving-cert\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135484 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135319 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/090cb996-2729-46af-adac-941ef69bc0f8-serving-cert\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.135484 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135409 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xpqs\" (UniqueName: \"kubernetes.io/projected/090cb996-2729-46af-adac-941ef69bc0f8-kube-api-access-7xpqs\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.135484 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135438 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ca8083f-bc69-461c-80e3-d5abc98eb611-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135690 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135492 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ca8083f-bc69-461c-80e3-d5abc98eb611-tmp\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135690 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.135551 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls podName:66fa2c63-4c75-49d1-8dd7-37bb9caa975b nodeName:}" failed. No retries permitted until 2026-07-14 11:18:49.635532667 +0000 UTC m=+59.491366817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls") pod "cluster-samples-operator-65db858894-czxtw" (UID: "66fa2c63-4c75-49d1-8dd7-37bb9caa975b") : secret "samples-operator-tls" not found Jul 14 11:18:49.135690 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135554 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8ca8083f-bc69-461c-80e3-d5abc98eb611-snapshots\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.135953 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135929 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ca8083f-bc69-461c-80e3-d5abc98eb611-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.136060 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.135990 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ca8083f-bc69-461c-80e3-d5abc98eb611-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.136143 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.136125 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/090cb996-2729-46af-adac-941ef69bc0f8-trusted-ca\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.136478 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.136458 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/090cb996-2729-46af-adac-941ef69bc0f8-config\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.137805 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.137788 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/090cb996-2729-46af-adac-941ef69bc0f8-serving-cert\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.137857 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.137812 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8ca8083f-bc69-461c-80e3-d5abc98eb611-serving-cert\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.146057 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.146030 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gbm5\" (UniqueName: \"kubernetes.io/projected/8ca8083f-bc69-461c-80e3-d5abc98eb611-kube-api-access-2gbm5\") pod \"insights-operator-68d9dd94cf-wl2m9\" (UID: \"8ca8083f-bc69-461c-80e3-d5abc98eb611\") " pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.146292 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.146272 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xpqs\" (UniqueName: \"kubernetes.io/projected/090cb996-2729-46af-adac-941ef69bc0f8-kube-api-access-7xpqs\") pod \"console-operator-7866497494-l8hd4\" (UID: \"090cb996-2729-46af-adac-941ef69bc0f8\") " pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.146439 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.146412 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq82d\" (UniqueName: \"kubernetes.io/projected/ac55edc5-a5d4-4283-8b76-00d0fd56ec8e-kube-api-access-xq82d\") pod \"volume-data-source-validator-78495597f6-bdtbp\" (UID: \"ac55edc5-a5d4-4283-8b76-00d0fd56ec8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" Jul 14 11:18:49.146589 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.146572 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvbv\" (UniqueName: \"kubernetes.io/projected/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-kube-api-access-wjvbv\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.296157 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.296124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" Jul 14 11:18:49.305719 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.305688 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" Jul 14 11:18:49.312450 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.312426 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:49.440942 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.440918 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp"] Jul 14 11:18:49.443420 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:49.443393 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac55edc5_a5d4_4283_8b76_00d0fd56ec8e.slice/crio-aa326324020312e38c6ede4fd1bf345e301ddc56a44f84f0399aa2dfa2423da8 WatchSource:0}: Error finding container aa326324020312e38c6ede4fd1bf345e301ddc56a44f84f0399aa2dfa2423da8: Status 404 returned error can't find the container with id aa326324020312e38c6ede4fd1bf345e301ddc56a44f84f0399aa2dfa2423da8 Jul 14 11:18:49.539603 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.539570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.539754 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.539634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:49.539754 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.539684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:49.539840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.539750 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:50.539725371 +0000 UTC m=+60.395559537 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : configmap references non-existent config key: service-ca.crt Jul 14 11:18:49.539840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.539775 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 11:18:49.539840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.539777 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:49.539840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.539817 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls podName:17070e67-dd32-4d46-a9c1-7e2d634e2ccd nodeName:}" failed. No retries permitted until 2026-07-14 11:18:50.539809865 +0000 UTC m=+60.395644014 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-rjc8c" (UID: "17070e67-dd32-4d46-a9c1-7e2d634e2ccd") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:49.539840 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.539834 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:50.539828924 +0000 UTC m=+60.395663073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : secret "router-metrics-certs-default" not found Jul 14 11:18:49.640620 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.640547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:49.640746 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.640698 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 11:18:49.640784 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:49.640759 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls podName:66fa2c63-4c75-49d1-8dd7-37bb9caa975b nodeName:}" failed. No retries permitted until 2026-07-14 11:18:50.640744251 +0000 UTC m=+60.496578401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls") pod "cluster-samples-operator-65db858894-czxtw" (UID: "66fa2c63-4c75-49d1-8dd7-37bb9caa975b") : secret "samples-operator-tls" not found Jul 14 11:18:49.666931 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.666898 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wl2m9"] Jul 14 11:18:49.667785 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.667763 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-l8hd4"] Jul 14 11:18:49.669618 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:49.669592 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca8083f_bc69_461c_80e3_d5abc98eb611.slice/crio-7edab0ae92f50b13d6c87dd7ef482e071f6850006baa2f6c228febf7a3b23bd5 WatchSource:0}: Error finding container 7edab0ae92f50b13d6c87dd7ef482e071f6850006baa2f6c228febf7a3b23bd5: Status 404 returned error can't find the container with id 7edab0ae92f50b13d6c87dd7ef482e071f6850006baa2f6c228febf7a3b23bd5 Jul 14 11:18:49.670315 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:49.670248 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090cb996_2729_46af_adac_941ef69bc0f8.slice/crio-23592738b985df6841c8b003bc5abef5a8ff4ba5988af1f5f1df6f011c08f0d1 WatchSource:0}: Error finding container 23592738b985df6841c8b003bc5abef5a8ff4ba5988af1f5f1df6f011c08f0d1: Status 404 returned error can't find the container with id 23592738b985df6841c8b003bc5abef5a8ff4ba5988af1f5f1df6f011c08f0d1 Jul 14 11:18:49.981838 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.981743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" event={"ID":"ac55edc5-a5d4-4283-8b76-00d0fd56ec8e","Type":"ContainerStarted","Data":"aa326324020312e38c6ede4fd1bf345e301ddc56a44f84f0399aa2dfa2423da8"} Jul 14 11:18:49.982964 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.982940 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" event={"ID":"8ca8083f-bc69-461c-80e3-d5abc98eb611","Type":"ContainerStarted","Data":"7edab0ae92f50b13d6c87dd7ef482e071f6850006baa2f6c228febf7a3b23bd5"} Jul 14 11:18:49.984306 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:49.984287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-l8hd4" event={"ID":"090cb996-2729-46af-adac-941ef69bc0f8","Type":"ContainerStarted","Data":"23592738b985df6841c8b003bc5abef5a8ff4ba5988af1f5f1df6f011c08f0d1"} Jul 14 11:18:50.552460 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:50.552364 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.556301 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:50.556367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.556385 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls podName:17070e67-dd32-4d46-a9c1-7e2d634e2ccd nodeName:}" failed. No retries permitted until 2026-07-14 11:18:52.556362366 +0000 UTC m=+62.412196526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-rjc8c" (UID: "17070e67-dd32-4d46-a9c1-7e2d634e2ccd") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.556467 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:50.556478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.556528 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:52.556503099 +0000 UTC m=+62.412337250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : secret "router-metrics-certs-default" not found Jul 14 11:18:50.557050 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.556582 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:52.556565309 +0000 UTC m=+62.412399472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : configmap references non-existent config key: service-ca.crt Jul 14 11:18:50.657080 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:50.657034 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:50.657202 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.657181 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 11:18:50.657316 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:50.657272 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls podName:66fa2c63-4c75-49d1-8dd7-37bb9caa975b nodeName:}" failed. No retries permitted until 2026-07-14 11:18:52.657235678 +0000 UTC m=+62.513069834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls") pod "cluster-samples-operator-65db858894-czxtw" (UID: "66fa2c63-4c75-49d1-8dd7-37bb9caa975b") : secret "samples-operator-tls" not found Jul 14 11:18:50.988563 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:50.988476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" event={"ID":"ac55edc5-a5d4-4283-8b76-00d0fd56ec8e","Type":"ContainerStarted","Data":"d3e55a3460fe1476ebaa80547333162afba6441d5699fa38834c240cdb20c097"} Jul 14 11:18:51.006840 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:51.006799 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdtbp" podStartSLOduration=1.823839387 podStartE2EDuration="3.006786108s" totalImagesPullingTime="1.182946721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:51.005943791 +0000 UTC m=+60.861777962" watchObservedRunningTime="2026-07-14 11:18:51.006786108 +0000 UTC m=+60.862620278" Jul 14 11:18:51.881207 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:51.881180 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngnhm" Jul 14 11:18:52.576658 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.576625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.576684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.576736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.576780 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.576813 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.576842 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.576826912 +0000 UTC m=+66.432661061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : secret "router-metrics-certs-default" not found Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.576862 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls podName:17070e67-dd32-4d46-a9c1-7e2d634e2ccd nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.576849452 +0000 UTC m=+66.432683620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-rjc8c" (UID: "17070e67-dd32-4d46-a9c1-7e2d634e2ccd") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:52.577065 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.576877 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.576869105 +0000 UTC m=+66.432703254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : configmap references non-existent config key: service-ca.crt Jul 14 11:18:52.677806 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.677779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:52.677935 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.677919 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 11:18:52.677983 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:52.677974 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls podName:66fa2c63-4c75-49d1-8dd7-37bb9caa975b nodeName:}" failed. No retries permitted until 2026-07-14 11:18:56.677959766 +0000 UTC m=+66.533793915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls") pod "cluster-samples-operator-65db858894-czxtw" (UID: "66fa2c63-4c75-49d1-8dd7-37bb9caa975b") : secret "samples-operator-tls" not found Jul 14 11:18:52.886511 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.886482 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns"] Jul 14 11:18:52.888370 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.888354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:52.891221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.891171 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 11:18:52.891221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.891176 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:52.891221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.891215 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 11:18:52.892916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.892901 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hgr7c\"" Jul 14 11:18:52.892916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.892908 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:52.899442 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.899422 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns"] Jul 14 11:18:52.980676 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.980641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:52.980824 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.980808 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:52.980884 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.980839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhdl4\" (UniqueName: \"kubernetes.io/projected/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-kube-api-access-vhdl4\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:52.996894 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.996864 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" event={"ID":"8ca8083f-bc69-461c-80e3-d5abc98eb611","Type":"ContainerStarted","Data":"e1409590465b5e3d29adbb6f67336300538ca6b21e9ba8c78c41964d88a21863"} Jul 14 11:18:52.998575 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.998556 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/0.log" Jul 14 11:18:52.998657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.998595 2568 generic.go:356] "Generic (PLEG): container finished" podID="090cb996-2729-46af-adac-941ef69bc0f8" containerID="b8cc8f4fa151067e643c0632864e3a9d15d5e6e7bcec9e9d72c4e28b39760ee1" exitCode=255 Jul 14 11:18:52.998657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.998625 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-l8hd4" event={"ID":"090cb996-2729-46af-adac-941ef69bc0f8","Type":"ContainerDied","Data":"b8cc8f4fa151067e643c0632864e3a9d15d5e6e7bcec9e9d72c4e28b39760ee1"} Jul 14 11:18:52.998824 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:52.998810 2568 scope.go:117] "RemoveContainer" containerID="b8cc8f4fa151067e643c0632864e3a9d15d5e6e7bcec9e9d72c4e28b39760ee1" Jul 14 11:18:53.036990 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.036933 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" podStartSLOduration=1.975973169 podStartE2EDuration="5.036911557s" totalImagesPullingTime="3.060938388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:53.016342561 +0000 UTC m=+62.872176732" watchObservedRunningTime="2026-07-14 11:18:53.036911557 +0000 UTC m=+62.892745732" Jul 14 11:18:53.082020 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.081962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.082538 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.082283 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhdl4\" (UniqueName: \"kubernetes.io/projected/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-kube-api-access-vhdl4\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.082538 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.082412 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.082943 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.082921 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.084552 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.084533 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.090111 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.090090 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhdl4\" (UniqueName: \"kubernetes.io/projected/c6eb804d-f51c-45b4-a39d-e56acbe6d53f-kube-api-access-vhdl4\") pod \"kube-storage-version-migrator-operator-799cb7f958-l2cns\" (UID: \"c6eb804d-f51c-45b4-a39d-e56acbe6d53f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.229910 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.229878 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" Jul 14 11:18:53.366102 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.366069 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns"] Jul 14 11:18:53.369484 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:53.369456 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6eb804d_f51c_45b4_a39d_e56acbe6d53f.slice/crio-d932c6221db60759b76326c37e6615a20d8acb2e2e500478ba3fc61d438a7152 WatchSource:0}: Error finding container d932c6221db60759b76326c37e6615a20d8acb2e2e500478ba3fc61d438a7152: Status 404 returned error can't find the container with id d932c6221db60759b76326c37e6615a20d8acb2e2e500478ba3fc61d438a7152 Jul 14 11:18:53.780687 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.780655 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z5j9x"] Jul 14 11:18:53.783670 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.783649 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.793800 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.793772 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 11:18:53.793929 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.793836 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 11:18:53.793929 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.793840 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xrcwt\"" Jul 14 11:18:53.808748 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.808725 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z5j9x"] Jul 14 11:18:53.889508 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.889468 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.889691 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.889516 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.889691 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.889536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a2a6105-4b48-40b0-8f75-2023d94157fa-crio-socket\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.889691 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.889617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pfkn\" (UniqueName: \"kubernetes.io/projected/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-api-access-6pfkn\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.889691 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.889684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6105-4b48-40b0-8f75-2023d94157fa-data-volume\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990066 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990031 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pfkn\" (UniqueName: \"kubernetes.io/projected/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-api-access-6pfkn\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990243 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6105-4b48-40b0-8f75-2023d94157fa-data-volume\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990243 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990343 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990343 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990301 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a2a6105-4b48-40b0-8f75-2023d94157fa-crio-socket\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990343 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:53.990327 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:18:53.990427 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:53.990395 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:18:54.490377056 +0000 UTC m=+64.346211224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : configmap "kube-rbac-proxy" not found Jul 14 11:18:53.990427 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990413 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a2a6105-4b48-40b0-8f75-2023d94157fa-crio-socket\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990493 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:53.990448 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:18:53.990523 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:53.990502 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6105-4b48-40b0-8f75-2023d94157fa-data-volume\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:53.990523 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:53.990509 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:18:54.490490763 +0000 UTC m=+64.346324940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : secret "insights-runtime-extractor-tls" not found Jul 14 11:18:54.001966 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.001938 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pfkn\" (UniqueName: \"kubernetes.io/projected/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-api-access-6pfkn\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:54.002923 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.002871 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/1.log" Jul 14 11:18:54.003304 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.003289 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/0.log" Jul 14 11:18:54.003374 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.003322 2568 generic.go:356] "Generic (PLEG): container finished" podID="090cb996-2729-46af-adac-941ef69bc0f8" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" exitCode=255 Jul 14 11:18:54.003423 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.003383 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-l8hd4" event={"ID":"090cb996-2729-46af-adac-941ef69bc0f8","Type":"ContainerDied","Data":"176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298"} Jul 14 11:18:54.003464 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.003434 2568 scope.go:117] "RemoveContainer" containerID="b8cc8f4fa151067e643c0632864e3a9d15d5e6e7bcec9e9d72c4e28b39760ee1" Jul 14 11:18:54.003619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.003602 2568 scope.go:117] "RemoveContainer" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" Jul 14 11:18:54.003826 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:54.003801 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-l8hd4_openshift-console-operator(090cb996-2729-46af-adac-941ef69bc0f8)\"" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podUID="090cb996-2729-46af-adac-941ef69bc0f8" Jul 14 11:18:54.004827 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.004626 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" event={"ID":"c6eb804d-f51c-45b4-a39d-e56acbe6d53f","Type":"ContainerStarted","Data":"d932c6221db60759b76326c37e6615a20d8acb2e2e500478ba3fc61d438a7152"} Jul 14 11:18:54.495425 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.495382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:54.495709 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.495436 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:54.495709 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:54.495514 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:18:54.495709 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:54.495538 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:18:54.495709 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:54.495585 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:18:55.49556738 +0000 UTC m=+65.351401548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : configmap "kube-rbac-proxy" not found Jul 14 11:18:54.495890 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:54.495737 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:18:55.495717258 +0000 UTC m=+65.351551414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : secret "insights-runtime-extractor-tls" not found Jul 14 11:18:54.882377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.882349 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g"] Jul 14 11:18:54.885339 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.885320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:54.888619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.888587 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9kg7s\"" Jul 14 11:18:54.888619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.888605 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 11:18:54.888802 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.888746 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:54.889988 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.889967 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 11:18:54.890104 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.890040 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:54.898279 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.898225 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g"] Jul 14 11:18:54.977448 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:54.977406 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t2jvz_445082e1-8f80-4e41-8164-2a8ce6a18b10/dns-node-resolver/0.log" Jul 14 11:18:55.001360 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.001322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2163c957-e628-40c6-b7ca-57601411acfe-config\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.001537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.001396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2163c957-e628-40c6-b7ca-57601411acfe-serving-cert\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.001537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.001439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzg4j\" (UniqueName: \"kubernetes.io/projected/2163c957-e628-40c6-b7ca-57601411acfe-kube-api-access-kzg4j\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.009773 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.009743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/1.log" Jul 14 11:18:55.010163 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.010129 2568 scope.go:117] "RemoveContainer" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" Jul 14 11:18:55.010346 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:55.010326 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-l8hd4_openshift-console-operator(090cb996-2729-46af-adac-941ef69bc0f8)\"" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podUID="090cb996-2729-46af-adac-941ef69bc0f8" Jul 14 11:18:55.102477 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.102428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzg4j\" (UniqueName: \"kubernetes.io/projected/2163c957-e628-40c6-b7ca-57601411acfe-kube-api-access-kzg4j\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.102660 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.102613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2163c957-e628-40c6-b7ca-57601411acfe-config\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.102728 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.102666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2163c957-e628-40c6-b7ca-57601411acfe-serving-cert\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.103362 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.103340 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2163c957-e628-40c6-b7ca-57601411acfe-config\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.105183 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.105160 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2163c957-e628-40c6-b7ca-57601411acfe-serving-cert\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.113016 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.112981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzg4j\" (UniqueName: \"kubernetes.io/projected/2163c957-e628-40c6-b7ca-57601411acfe-kube-api-access-kzg4j\") pod \"service-ca-operator-7695fc6c8c-42z4g\" (UID: \"2163c957-e628-40c6-b7ca-57601411acfe\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.195297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.195190 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" Jul 14 11:18:55.506457 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.506379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:55.506626 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:55.506498 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:18:55.506626 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.506527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:55.506626 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:55.506562 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:18:57.50654334 +0000 UTC m=+67.362377491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : configmap "kube-rbac-proxy" not found Jul 14 11:18:55.506781 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:55.506661 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:18:55.506781 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:55.506707 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:18:57.506695472 +0000 UTC m=+67.362529637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : secret "insights-runtime-extractor-tls" not found Jul 14 11:18:55.508498 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.508476 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g"] Jul 14 11:18:55.510789 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:55.510763 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2163c957_e628_40c6_b7ca_57601411acfe.slice/crio-5fdae2d4956b64a1f2a72027e1b4d5008205e4f48f66302379ddaa9aa477976a WatchSource:0}: Error finding container 5fdae2d4956b64a1f2a72027e1b4d5008205e4f48f66302379ddaa9aa477976a: Status 404 returned error can't find the container with id 5fdae2d4956b64a1f2a72027e1b4d5008205e4f48f66302379ddaa9aa477976a Jul 14 11:18:55.977521 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:55.977490 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hxzbm_49806619-4c59-4b80-b24f-22162de9877b/node-ca/0.log" Jul 14 11:18:56.015000 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.014894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" event={"ID":"c6eb804d-f51c-45b4-a39d-e56acbe6d53f","Type":"ContainerStarted","Data":"356d4521a1287e2970d7e4bdd50a0022488cbc73d91ab85c5ac874c6be0242c0"} Jul 14 11:18:56.016047 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.016019 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" event={"ID":"2163c957-e628-40c6-b7ca-57601411acfe","Type":"ContainerStarted","Data":"5fdae2d4956b64a1f2a72027e1b4d5008205e4f48f66302379ddaa9aa477976a"} Jul 14 11:18:56.034704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.034655 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" podStartSLOduration=1.9665121399999999 podStartE2EDuration="4.034641157s" totalImagesPullingTime="2.068129017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:56.033494485 +0000 UTC m=+65.889328660" watchObservedRunningTime="2026-07-14 11:18:56.034641157 +0000 UTC m=+65.890475327" Jul 14 11:18:56.075281 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.073215 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr"] Jul 14 11:18:56.077585 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.077554 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" Jul 14 11:18:56.080513 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.080492 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vk4fn\"" Jul 14 11:18:56.083053 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.083032 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr"] Jul 14 11:18:56.213352 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.213315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbgv4\" (UniqueName: \"kubernetes.io/projected/0176c376-9e07-4e90-b87f-5ea1f8cca24c-kube-api-access-wbgv4\") pod \"network-check-source-869dcfcdcd-6hbzr\" (UID: \"0176c376-9e07-4e90-b87f-5ea1f8cca24c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" Jul 14 11:18:56.314447 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.314416 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbgv4\" (UniqueName: \"kubernetes.io/projected/0176c376-9e07-4e90-b87f-5ea1f8cca24c-kube-api-access-wbgv4\") pod \"network-check-source-869dcfcdcd-6hbzr\" (UID: \"0176c376-9e07-4e90-b87f-5ea1f8cca24c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" Jul 14 11:18:56.324570 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.324548 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 11:18:56.335319 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.335299 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 11:18:56.346012 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.345984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbgv4\" (UniqueName: \"kubernetes.io/projected/0176c376-9e07-4e90-b87f-5ea1f8cca24c-kube-api-access-wbgv4\") pod \"network-check-source-869dcfcdcd-6hbzr\" (UID: \"0176c376-9e07-4e90-b87f-5ea1f8cca24c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" Jul 14 11:18:56.387143 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.387107 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" Jul 14 11:18:56.516950 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.516912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:18:56.516950 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.516952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:56.519822 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.519806 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 11:18:56.519896 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.519877 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 11:18:56.527818 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.527795 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 11:18:56.527929 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.527855 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs podName:ecc55306-170e-463f-b3e5-263e3201fc1c nodeName:}" failed. No retries permitted until 2026-07-14 11:20:00.527840291 +0000 UTC m=+130.383674440 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs") pod "network-metrics-daemon-8dclj" (UID: "ecc55306-170e-463f-b3e5-263e3201fc1c") : secret "metrics-daemon-secret" not found Jul 14 11:18:56.529944 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.529918 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e1eb7e2-1e05-4af6-93f1-894e406169c9-original-pull-secret\") pod \"global-pull-secret-syncer-2qzv4\" (UID: \"7e1eb7e2-1e05-4af6-93f1-894e406169c9\") " pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:56.534939 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.534909 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr"] Jul 14 11:18:56.537622 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:56.537598 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0176c376_9e07_4e90_b87f_5ea1f8cca24c.slice/crio-4d2c052778d5460aa0a3a79394a8854d2c35c679fb53b55c08ce3b8e81207bd8 WatchSource:0}: Error finding container 4d2c052778d5460aa0a3a79394a8854d2c35c679fb53b55c08ce3b8e81207bd8: Status 404 returned error can't find the container with id 4d2c052778d5460aa0a3a79394a8854d2c35c679fb53b55c08ce3b8e81207bd8 Jul 14 11:18:56.576980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.576907 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9jdh_648326eb-7168-4bdb-a3e0-e7b510948491/serve-healthcheck-canary/0.log" Jul 14 11:18:56.594050 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.594025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2qzv4" Jul 14 11:18:56.617490 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.617459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:56.617616 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.617582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:18:56.617677 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.617610 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.61758707 +0000 UTC m=+74.473421231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : configmap references non-existent config key: service-ca.crt Jul 14 11:18:56.617805 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.617682 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:56.617805 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.617703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:18:56.617805 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.617735 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls podName:17070e67-dd32-4d46-a9c1-7e2d634e2ccd nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.617717365 +0000 UTC m=+74.473551542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-rjc8c" (UID: "17070e67-dd32-4d46-a9c1-7e2d634e2ccd") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:18:56.617805 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.617774 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 11:18:56.618082 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.617810 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.617798388 +0000 UTC m=+74.473632550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : secret "router-metrics-certs-default" not found Jul 14 11:18:56.618082 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.617860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:56.621688 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.621666 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bp2d\" (UniqueName: \"kubernetes.io/projected/c70ba287-c702-4f55-bc66-ed761b25607c-kube-api-access-2bp2d\") pod \"network-check-target-tffnh\" (UID: \"c70ba287-c702-4f55-bc66-ed761b25607c\") " pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:56.719117 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.719078 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:18:56.719284 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.719223 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 11:18:56.719344 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:56.719315 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls podName:66fa2c63-4c75-49d1-8dd7-37bb9caa975b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.719298826 +0000 UTC m=+74.575132975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls") pod "cluster-samples-operator-65db858894-czxtw" (UID: "66fa2c63-4c75-49d1-8dd7-37bb9caa975b") : secret "samples-operator-tls" not found Jul 14 11:18:56.750764 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.750736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2qzv4"] Jul 14 11:18:56.753311 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:56.753281 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1eb7e2_1e05_4af6_93f1_894e406169c9.slice/crio-9d002925a1d0a5a71c7b9c01a2e70f0a0d827c33b7971054205d21b0159d0d5b WatchSource:0}: Error finding container 9d002925a1d0a5a71c7b9c01a2e70f0a0d827c33b7971054205d21b0159d0d5b: Status 404 returned error can't find the container with id 9d002925a1d0a5a71c7b9c01a2e70f0a0d827c33b7971054205d21b0159d0d5b Jul 14 11:18:56.902930 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.902543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q2rq4\"" Jul 14 11:18:56.910715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.910689 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:18:56.968634 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.968328 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl"] Jul 14 11:18:56.975206 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.974606 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" Jul 14 11:18:56.977729 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.977703 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 14 11:18:56.978100 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.977756 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:56.978100 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.977755 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qsgdq\"" Jul 14 11:18:56.982460 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:56.982387 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl"] Jul 14 11:18:57.022559 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.022530 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2qzv4" event={"ID":"7e1eb7e2-1e05-4af6-93f1-894e406169c9","Type":"ContainerStarted","Data":"9d002925a1d0a5a71c7b9c01a2e70f0a0d827c33b7971054205d21b0159d0d5b"} Jul 14 11:18:57.025370 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.025301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" event={"ID":"0176c376-9e07-4e90-b87f-5ea1f8cca24c","Type":"ContainerStarted","Data":"4d2c052778d5460aa0a3a79394a8854d2c35c679fb53b55c08ce3b8e81207bd8"} Jul 14 11:18:57.056303 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.056273 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tffnh"] Jul 14 11:18:57.059648 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:18:57.059624 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70ba287_c702_4f55_bc66_ed761b25607c.slice/crio-be41a7a102ab84643e43c1285d5310abeb04ccb4ebb739e7c3d4a1aba4b79e5b WatchSource:0}: Error finding container be41a7a102ab84643e43c1285d5310abeb04ccb4ebb739e7c3d4a1aba4b79e5b: Status 404 returned error can't find the container with id be41a7a102ab84643e43c1285d5310abeb04ccb4ebb739e7c3d4a1aba4b79e5b Jul 14 11:18:57.123502 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.123459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj8ls\" (UniqueName: \"kubernetes.io/projected/0ce35024-2bd2-4602-ade2-222d50edcfb1-kube-api-access-mj8ls\") pod \"migrator-7f59b54f6c-qfbpl\" (UID: \"0ce35024-2bd2-4602-ade2-222d50edcfb1\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" Jul 14 11:18:57.224524 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.224430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj8ls\" (UniqueName: \"kubernetes.io/projected/0ce35024-2bd2-4602-ade2-222d50edcfb1-kube-api-access-mj8ls\") pod \"migrator-7f59b54f6c-qfbpl\" (UID: \"0ce35024-2bd2-4602-ade2-222d50edcfb1\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" Jul 14 11:18:57.258336 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.258303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj8ls\" (UniqueName: \"kubernetes.io/projected/0ce35024-2bd2-4602-ade2-222d50edcfb1-kube-api-access-mj8ls\") pod \"migrator-7f59b54f6c-qfbpl\" (UID: \"0ce35024-2bd2-4602-ade2-222d50edcfb1\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" Jul 14 11:18:57.290347 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.290312 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" Jul 14 11:18:57.528103 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.528017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:57.528103 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.528070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:18:57.528359 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:57.528163 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:18:57.528359 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:57.528191 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:18:57.528359 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:57.528238 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.528223833 +0000 UTC m=+71.384057982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : secret "insights-runtime-extractor-tls" not found Jul 14 11:18:57.528359 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:57.528279 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.528246716 +0000 UTC m=+71.384080867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : configmap "kube-rbac-proxy" not found Jul 14 11:18:57.795427 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:57.795401 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl"] Jul 14 11:18:58.033988 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:58.032201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" event={"ID":"0ce35024-2bd2-4602-ade2-222d50edcfb1","Type":"ContainerStarted","Data":"3c21f637e29cd1fb05c7833a08e71ce43821758898f55d84cf29d082a274f1c6"} Jul 14 11:18:58.035860 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:58.035575 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" event={"ID":"2163c957-e628-40c6-b7ca-57601411acfe","Type":"ContainerStarted","Data":"f1048c140124d71184fa1c138d0a53e59c7811db9add70e6e6e2b1c9121ea056"} Jul 14 11:18:58.039780 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:58.037949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tffnh" event={"ID":"c70ba287-c702-4f55-bc66-ed761b25607c","Type":"ContainerStarted","Data":"be41a7a102ab84643e43c1285d5310abeb04ccb4ebb739e7c3d4a1aba4b79e5b"} Jul 14 11:18:58.055906 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:58.055371 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" podStartSLOduration=1.85972906 podStartE2EDuration="4.055355931s" totalImagesPullingTime="2.195626871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:58.054538049 +0000 UTC m=+67.910372219" watchObservedRunningTime="2026-07-14 11:18:58.055355931 +0000 UTC m=+67.911190102" Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:58.843364 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:58.843443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:58.843524 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:58.843540 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-678db546b5-v268x: secret "image-registry-tls" not found Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:58.843571 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:58.843595 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls podName:ed757a72-785a-4619-b72d-aa643cf0ae32 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:30.843578347 +0000 UTC m=+100.699412518 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls") pod "image-registry-678db546b5-v268x" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32") : secret "image-registry-tls" not found Jul 14 11:18:58.844329 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:58.843610 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls podName:53beda37-da6d-4cff-88c6-8341416a83de nodeName:}" failed. No retries permitted until 2026-07-14 11:19:30.843602253 +0000 UTC m=+100.699436403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls") pod "dns-default-mq2gr" (UID: "53beda37-da6d-4cff-88c6-8341416a83de") : secret "dns-default-metrics-tls" not found Jul 14 11:18:59.312674 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:59.312642 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:18:59.313161 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:59.313145 2568 scope.go:117] "RemoveContainer" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" Jul 14 11:18:59.313389 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:18:59.313363 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-l8hd4_openshift-console-operator(090cb996-2729-46af-adac-941ef69bc0f8)\"" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podUID="090cb996-2729-46af-adac-941ef69bc0f8" Jul 14 11:18:59.313809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:18:59.313789 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:19:00.047858 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:00.047816 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" event={"ID":"0ce35024-2bd2-4602-ade2-222d50edcfb1","Type":"ContainerStarted","Data":"28bb60057b9893070ba47fa2a8feb314f8f1a1524dca7cde7ef4225e1bfcdfa8"} Jul 14 11:19:00.048164 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:00.048146 2568 scope.go:117] "RemoveContainer" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" Jul 14 11:19:00.048364 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:00.048344 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-l8hd4_openshift-console-operator(090cb996-2729-46af-adac-941ef69bc0f8)\"" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podUID="090cb996-2729-46af-adac-941ef69bc0f8" Jul 14 11:19:01.577036 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:01.576689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:01.577036 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:01.576757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:01.577036 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:01.576823 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:19:01.577036 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:01.576900 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:19:09.576877525 +0000 UTC m=+79.432711687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : configmap "kube-rbac-proxy" not found Jul 14 11:19:01.577036 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:01.576938 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:01.577036 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:01.576991 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:19:09.57697631 +0000 UTC m=+79.432810473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:03.059321 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.059279 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2qzv4" event={"ID":"7e1eb7e2-1e05-4af6-93f1-894e406169c9","Type":"ContainerStarted","Data":"9757d9b547bc0d993a8ee88c870490a9ad4d43c32b09413c211d4848f90b98ca"} Jul 14 11:19:03.060794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.060761 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" event={"ID":"0176c376-9e07-4e90-b87f-5ea1f8cca24c","Type":"ContainerStarted","Data":"bb1949da33bb622df3a5afb7d502abbcb2bad64d6659b1b445fd9ec0ccb47c1c"} Jul 14 11:19:03.062307 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.062281 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" event={"ID":"0ce35024-2bd2-4602-ade2-222d50edcfb1","Type":"ContainerStarted","Data":"15ea8bc99292c7c6c2a11f4517f97bd8fcbec314f3a46cda08dcb420c4d1a1db"} Jul 14 11:19:03.063635 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.063614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tffnh" event={"ID":"c70ba287-c702-4f55-bc66-ed761b25607c","Type":"ContainerStarted","Data":"d5dd2d573745e8da2bd82e78407bd4d00ca8e1e432c0148dd010242aed75f1ec"} Jul 14 11:19:03.063756 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.063741 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:19:03.078383 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.078339 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2qzv4" podStartSLOduration=66.498541666 podStartE2EDuration="1m12.078294562s" totalImagesPullingTime="5.579752896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:03.07716189 +0000 UTC m=+72.932996062" watchObservedRunningTime="2026-07-14 11:19:03.078294562 +0000 UTC m=+72.934160569" Jul 14 11:19:03.098773 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.098729 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6hbzr" podStartSLOduration=1.308886651 podStartE2EDuration="7.098716565s" totalImagesPullingTime="5.789829914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:03.096880261 +0000 UTC m=+72.952714431" watchObservedRunningTime="2026-07-14 11:19:03.098716565 +0000 UTC m=+72.954550730" Jul 14 11:19:03.117995 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.117946 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qfbpl" podStartSLOduration=5.138872325 podStartE2EDuration="7.117932s" totalImagesPullingTime="1.979059675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:03.116321305 +0000 UTC m=+72.972155477" watchObservedRunningTime="2026-07-14 11:19:03.117932 +0000 UTC m=+72.973766173" Jul 14 11:19:03.136029 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:03.135990 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tffnh" podStartSLOduration=66.858451472 podStartE2EDuration="1m12.135978013s" totalImagesPullingTime="5.277526541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:03.135104255 +0000 UTC m=+72.990938429" watchObservedRunningTime="2026-07-14 11:19:03.135978013 +0000 UTC m=+72.991812183" Jul 14 11:19:04.705761 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:04.705725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:04.705784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.705863 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.705877 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.705905 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:20.705893167 +0000 UTC m=+90.561727317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : secret "router-metrics-certs-default" not found Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.705927 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls podName:17070e67-dd32-4d46-a9c1-7e2d634e2ccd nodeName:}" failed. No retries permitted until 2026-07-14 11:19:20.705915046 +0000 UTC m=+90.561749200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-rjc8c" (UID: "17070e67-dd32-4d46-a9c1-7e2d634e2ccd") : secret "cluster-monitoring-operator-tls" not found Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:04.705987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:04.706152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.706072 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle podName:00628d94-4a03-456d-8c41-0ab813236ad1 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:20.706064342 +0000 UTC m=+90.561898492 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle") pod "router-default-794fd9f96d-dt6lg" (UID: "00628d94-4a03-456d-8c41-0ab813236ad1") : configmap references non-existent config key: service-ca.crt Jul 14 11:19:04.806564 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:04.806535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:19:04.806721 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.806686 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 11:19:04.806756 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:04.806743 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls podName:66fa2c63-4c75-49d1-8dd7-37bb9caa975b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:20.806729508 +0000 UTC m=+90.662563657 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls") pod "cluster-samples-operator-65db858894-czxtw" (UID: "66fa2c63-4c75-49d1-8dd7-37bb9caa975b") : secret "samples-operator-tls" not found Jul 14 11:19:09.648895 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:09.648852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:09.648895 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:09.648902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:09.649354 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:09.648988 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:09.649354 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:09.649071 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls podName:6a2a6105-4b48-40b0-8f75-2023d94157fa nodeName:}" failed. No retries permitted until 2026-07-14 11:19:25.64905366 +0000 UTC m=+95.504887815 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z5j9x" (UID: "6a2a6105-4b48-40b0-8f75-2023d94157fa") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:09.649454 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:09.649434 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a2a6105-4b48-40b0-8f75-2023d94157fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:10.674273 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:10.674217 2568 scope.go:117] "RemoveContainer" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" Jul 14 11:19:11.091914 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:11.091888 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:19:11.092337 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:11.092322 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/1.log" Jul 14 11:19:11.092399 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:11.092354 2568 generic.go:356] "Generic (PLEG): container finished" podID="090cb996-2729-46af-adac-941ef69bc0f8" containerID="e4b0aebc7ef6eec77a324e1290a9501bb4daadc0e0f0a67520c30d264f71ac9e" exitCode=255 Jul 14 11:19:11.092449 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:11.092393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-l8hd4" event={"ID":"090cb996-2729-46af-adac-941ef69bc0f8","Type":"ContainerDied","Data":"e4b0aebc7ef6eec77a324e1290a9501bb4daadc0e0f0a67520c30d264f71ac9e"} Jul 14 11:19:11.092499 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:11.092469 2568 scope.go:117] "RemoveContainer" containerID="176e96016aebec6e8bb117072b77ee77bdc10d5aadf53c65dcfbef3cc209f298" Jul 14 11:19:11.092818 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:11.092798 2568 scope.go:117] "RemoveContainer" containerID="e4b0aebc7ef6eec77a324e1290a9501bb4daadc0e0f0a67520c30d264f71ac9e" Jul 14 11:19:11.092979 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:11.092958 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-l8hd4_openshift-console-operator(090cb996-2729-46af-adac-941ef69bc0f8)\"" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podUID="090cb996-2729-46af-adac-941ef69bc0f8" Jul 14 11:19:12.102376 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:12.102275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:19:19.313408 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:19.313374 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:19:19.313803 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:19.313438 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:19:19.313803 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:19.313766 2568 scope.go:117] "RemoveContainer" containerID="e4b0aebc7ef6eec77a324e1290a9501bb4daadc0e0f0a67520c30d264f71ac9e" Jul 14 11:19:19.313932 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:19:19.313910 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-l8hd4_openshift-console-operator(090cb996-2729-46af-adac-941ef69bc0f8)\"" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podUID="090cb996-2729-46af-adac-941ef69bc0f8" Jul 14 11:19:20.739963 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.739928 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:19:20.740366 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.739984 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:20.740366 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.740040 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:20.740736 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.740712 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00628d94-4a03-456d-8c41-0ab813236ad1-service-ca-bundle\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:20.742568 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.742545 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00628d94-4a03-456d-8c41-0ab813236ad1-metrics-certs\") pod \"router-default-794fd9f96d-dt6lg\" (UID: \"00628d94-4a03-456d-8c41-0ab813236ad1\") " pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:20.742656 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.742575 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17070e67-dd32-4d46-a9c1-7e2d634e2ccd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-rjc8c\" (UID: \"17070e67-dd32-4d46-a9c1-7e2d634e2ccd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:19:20.841416 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.841389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:19:20.843823 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.843792 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66fa2c63-4c75-49d1-8dd7-37bb9caa975b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-czxtw\" (UID: \"66fa2c63-4c75-49d1-8dd7-37bb9caa975b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:19:20.988568 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.988539 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5jhwr\"" Jul 14 11:19:20.991973 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.991927 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mxzt8\"" Jul 14 11:19:20.995156 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:20.995142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" Jul 14 11:19:21.000642 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:21.000620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:21.119234 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:21.119191 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2xvvv\"" Jul 14 11:19:21.126698 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:21.126675 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" Jul 14 11:19:21.131007 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:21.130980 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c"] Jul 14 11:19:21.134565 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:19:21.134541 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17070e67_dd32_4d46_a9c1_7e2d634e2ccd.slice/crio-48ed50d9a341738f82cc3af3c878ac7d305b3668665f01cd53ab9b07e95f550c WatchSource:0}: Error finding container 48ed50d9a341738f82cc3af3c878ac7d305b3668665f01cd53ab9b07e95f550c: Status 404 returned error can't find the container with id 48ed50d9a341738f82cc3af3c878ac7d305b3668665f01cd53ab9b07e95f550c Jul 14 11:19:21.152512 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:21.152479 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-794fd9f96d-dt6lg"] Jul 14 11:19:21.154692 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:19:21.154660 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00628d94_4a03_456d_8c41_0ab813236ad1.slice/crio-b698eb062e0bacdc78caca08f0570e5c58f9f21c551b34eb604c70fdc04de213 WatchSource:0}: Error finding container b698eb062e0bacdc78caca08f0570e5c58f9f21c551b34eb604c70fdc04de213: Status 404 returned error can't find the container with id b698eb062e0bacdc78caca08f0570e5c58f9f21c551b34eb604c70fdc04de213 Jul 14 11:19:21.258662 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:21.258640 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw"] Jul 14 11:19:22.138811 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:22.138281 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" event={"ID":"00628d94-4a03-456d-8c41-0ab813236ad1","Type":"ContainerStarted","Data":"86496f519827eb27aaf98abf00f87f106ea25a7df278233953307ee674ee6e4a"} Jul 14 11:19:22.138811 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:22.138351 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" event={"ID":"00628d94-4a03-456d-8c41-0ab813236ad1","Type":"ContainerStarted","Data":"b698eb062e0bacdc78caca08f0570e5c58f9f21c551b34eb604c70fdc04de213"} Jul 14 11:19:22.139778 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:22.139743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" event={"ID":"17070e67-dd32-4d46-a9c1-7e2d634e2ccd","Type":"ContainerStarted","Data":"48ed50d9a341738f82cc3af3c878ac7d305b3668665f01cd53ab9b07e95f550c"} Jul 14 11:19:22.141103 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:22.141070 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" event={"ID":"66fa2c63-4c75-49d1-8dd7-37bb9caa975b","Type":"ContainerStarted","Data":"8a44660b061bfaeb9a050a4cb76fd00120605bf6e91cdfc5346d0b393771dfba"} Jul 14 11:19:22.176066 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:22.176004 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" podStartSLOduration=34.17598383 podStartE2EDuration="34.17598383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:22.175946807 +0000 UTC m=+92.031780982" watchObservedRunningTime="2026-07-14 11:19:22.17598383 +0000 UTC m=+92.031818002" Jul 14 11:19:23.001805 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:23.001766 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:23.004147 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:23.004127 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:23.145660 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:23.145616 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:23.147102 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:23.147076 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-794fd9f96d-dt6lg" Jul 14 11:19:24.150174 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:24.150136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" event={"ID":"17070e67-dd32-4d46-a9c1-7e2d634e2ccd","Type":"ContainerStarted","Data":"f2f24d18e8dba0e444ed49938bf00525997aba78e7c067446c70dd933f02b37e"} Jul 14 11:19:24.151907 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:24.151879 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" event={"ID":"66fa2c63-4c75-49d1-8dd7-37bb9caa975b","Type":"ContainerStarted","Data":"0bbd445510c2eee5e12db80490a72ee074fec5be902c453e78ef68c261ef901b"} Jul 14 11:19:24.152050 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:24.151929 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" event={"ID":"66fa2c63-4c75-49d1-8dd7-37bb9caa975b","Type":"ContainerStarted","Data":"63e46ec7296c4b248dc2714548e149eb5703b7dfea701d3ad799e056e770197a"} Jul 14 11:19:24.168822 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:24.168777 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-rjc8c" podStartSLOduration=33.793312582 podStartE2EDuration="36.168763587s" totalImagesPullingTime="2.375451005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:24.16773621 +0000 UTC m=+94.023570383" watchObservedRunningTime="2026-07-14 11:19:24.168763587 +0000 UTC m=+94.024597757" Jul 14 11:19:24.186995 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:24.186948 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-czxtw" podStartSLOduration=33.972155982 podStartE2EDuration="36.186930829s" totalImagesPullingTime="2.214774847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:24.185422492 +0000 UTC m=+94.041256664" watchObservedRunningTime="2026-07-14 11:19:24.186930829 +0000 UTC m=+94.042765000" Jul 14 11:19:25.687674 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:25.687640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:25.690100 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:25.690077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a2a6105-4b48-40b0-8f75-2023d94157fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5j9x\" (UID: \"6a2a6105-4b48-40b0-8f75-2023d94157fa\") " pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:25.896302 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:25.896249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z5j9x" Jul 14 11:19:26.024566 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:26.024536 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z5j9x"] Jul 14 11:19:26.027617 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:19:26.027592 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2a6105_4b48_40b0_8f75_2023d94157fa.slice/crio-098c5acb2eaa21069ac61d23b864caca5fb5446e27af37a665e64efe23e24181 WatchSource:0}: Error finding container 098c5acb2eaa21069ac61d23b864caca5fb5446e27af37a665e64efe23e24181: Status 404 returned error can't find the container with id 098c5acb2eaa21069ac61d23b864caca5fb5446e27af37a665e64efe23e24181 Jul 14 11:19:26.160011 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:26.159978 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5j9x" event={"ID":"6a2a6105-4b48-40b0-8f75-2023d94157fa","Type":"ContainerStarted","Data":"098c5acb2eaa21069ac61d23b864caca5fb5446e27af37a665e64efe23e24181"} Jul 14 11:19:30.936155 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.936118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:19:30.936535 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.936175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:19:30.938480 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.938452 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53beda37-da6d-4cff-88c6-8341416a83de-metrics-tls\") pod \"dns-default-mq2gr\" (UID: \"53beda37-da6d-4cff-88c6-8341416a83de\") " pod="openshift-dns/dns-default-mq2gr" Jul 14 11:19:30.938615 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.938595 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"image-registry-678db546b5-v268x\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:19:30.969952 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.969925 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rhbkm\"" Jul 14 11:19:30.975922 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.975905 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgqwk\"" Jul 14 11:19:30.977999 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.977983 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:19:30.984374 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:30.984358 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mq2gr" Jul 14 11:19:31.131045 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:31.131014 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-678db546b5-v268x"] Jul 14 11:19:31.151416 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:31.149105 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mq2gr"] Jul 14 11:19:31.180986 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:31.180955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5j9x" event={"ID":"6a2a6105-4b48-40b0-8f75-2023d94157fa","Type":"ContainerStarted","Data":"6710407c908c6724c8c949d8bd0d16fd660e20409b0bbbdd87a390d18beae7f4"} Jul 14 11:19:31.259557 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:19:31.259472 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded757a72_785a_4619_b72d_aa643cf0ae32.slice/crio-29ed3aa63fc4af0a8b56fa9012b39d26a8f7196ffbf0ef7f7f04276e4a31049e WatchSource:0}: Error finding container 29ed3aa63fc4af0a8b56fa9012b39d26a8f7196ffbf0ef7f7f04276e4a31049e: Status 404 returned error can't find the container with id 29ed3aa63fc4af0a8b56fa9012b39d26a8f7196ffbf0ef7f7f04276e4a31049e Jul 14 11:19:31.260421 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:19:31.260393 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53beda37_da6d_4cff_88c6_8341416a83de.slice/crio-0c61462c64787d47f3aee7e8f8fb447d2c3e2cd3c8a5e4b24c5ace00b4b7a099 WatchSource:0}: Error finding container 0c61462c64787d47f3aee7e8f8fb447d2c3e2cd3c8a5e4b24c5ace00b4b7a099: Status 404 returned error can't find the container with id 0c61462c64787d47f3aee7e8f8fb447d2c3e2cd3c8a5e4b24c5ace00b4b7a099 Jul 14 11:19:32.187978 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.187884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5j9x" event={"ID":"6a2a6105-4b48-40b0-8f75-2023d94157fa","Type":"ContainerStarted","Data":"2dcd1b892e40058bc33a3573b77fc950d3e6f78c818103c1cbe233eb859e63e0"} Jul 14 11:19:32.190015 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.189981 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678db546b5-v268x" event={"ID":"ed757a72-785a-4619-b72d-aa643cf0ae32","Type":"ContainerStarted","Data":"8b119ad3163a5b1c4d2f1901a2a1456e73aeaf36a8aaf1bfbaed3b52a0b23896"} Jul 14 11:19:32.190158 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.190036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678db546b5-v268x" event={"ID":"ed757a72-785a-4619-b72d-aa643cf0ae32","Type":"ContainerStarted","Data":"29ed3aa63fc4af0a8b56fa9012b39d26a8f7196ffbf0ef7f7f04276e4a31049e"} Jul 14 11:19:32.190989 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.190965 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:19:32.193210 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.193140 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mq2gr" event={"ID":"53beda37-da6d-4cff-88c6-8341416a83de","Type":"ContainerStarted","Data":"0c61462c64787d47f3aee7e8f8fb447d2c3e2cd3c8a5e4b24c5ace00b4b7a099"} Jul 14 11:19:32.219757 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.219688 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-678db546b5-v268x" podStartSLOduration=78.219670169 podStartE2EDuration="1m18.219670169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:32.217248325 +0000 UTC m=+102.073082499" watchObservedRunningTime="2026-07-14 11:19:32.219670169 +0000 UTC m=+102.075504346" Jul 14 11:19:32.602864 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.602828 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k7r8r"] Jul 14 11:19:32.617427 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.617394 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.620639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.620614 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5244g\"" Jul 14 11:19:32.620763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.620680 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 11:19:32.620763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.620704 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 11:19:32.620905 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.620892 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 11:19:32.622036 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.622014 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 11:19:32.755158 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755118 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-wtmp\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755171 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-textfile\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4464b705-8365-4a4f-ba5b-935b8c09e6ba-metrics-client-ca\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755321 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-tls\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755343 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755638 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755369 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-root\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755638 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755435 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-sys\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.755638 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.755479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj2bm\" (UniqueName: \"kubernetes.io/projected/4464b705-8365-4a4f-ba5b-935b8c09e6ba-kube-api-access-gj2bm\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-textfile\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857120 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4464b705-8365-4a4f-ba5b-935b8c09e6ba-metrics-client-ca\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857158 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-tls\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-root\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-root\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857529 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-textfile\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-sys\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj2bm\" (UniqueName: \"kubernetes.io/projected/4464b705-8365-4a4f-ba5b-935b8c09e6ba-kube-api-access-gj2bm\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.857735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857654 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-sys\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.858437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-wtmp\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.858437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857815 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.858437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857907 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-wtmp\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.858437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.857950 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4464b705-8365-4a4f-ba5b-935b8c09e6ba-metrics-client-ca\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.861287 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.861201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-tls\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.861410 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.861385 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4464b705-8365-4a4f-ba5b-935b8c09e6ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.869079 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.869054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj2bm\" (UniqueName: \"kubernetes.io/projected/4464b705-8365-4a4f-ba5b-935b8c09e6ba-kube-api-access-gj2bm\") pod \"node-exporter-k7r8r\" (UID: \"4464b705-8365-4a4f-ba5b-935b8c09e6ba\") " pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:32.929020 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:32.928988 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k7r8r" Jul 14 11:19:33.672360 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:33.672331 2568 scope.go:117] "RemoveContainer" containerID="e4b0aebc7ef6eec77a324e1290a9501bb4daadc0e0f0a67520c30d264f71ac9e" Jul 14 11:19:34.070598 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:34.070570 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tffnh" Jul 14 11:19:34.788072 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:19:34.788038 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4464b705_8365_4a4f_ba5b_935b8c09e6ba.slice/crio-617f7d76974a556b06e276c5a29a2d5e4b02148ddc03b097502dfa64926e4691 WatchSource:0}: Error finding container 617f7d76974a556b06e276c5a29a2d5e4b02148ddc03b097502dfa64926e4691: Status 404 returned error can't find the container with id 617f7d76974a556b06e276c5a29a2d5e4b02148ddc03b097502dfa64926e4691 Jul 14 11:19:35.207321 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.207294 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:19:35.207511 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.207403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-l8hd4" event={"ID":"090cb996-2729-46af-adac-941ef69bc0f8","Type":"ContainerStarted","Data":"0ec142821df7f00f12b6e875f8219f3087fced85e7b8324a58f617f6974e3a52"} Jul 14 11:19:35.207903 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.207877 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:19:35.209623 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.209598 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k7r8r" event={"ID":"4464b705-8365-4a4f-ba5b-935b8c09e6ba","Type":"ContainerStarted","Data":"617f7d76974a556b06e276c5a29a2d5e4b02148ddc03b097502dfa64926e4691"} Jul 14 11:19:35.212824 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.212793 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mq2gr" event={"ID":"53beda37-da6d-4cff-88c6-8341416a83de","Type":"ContainerStarted","Data":"f0fc53deef94cfe9ec3ebc746d0db16ff9cfc334752078a27b032bf1e99f3023"} Jul 14 11:19:35.212916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.212878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mq2gr" event={"ID":"53beda37-da6d-4cff-88c6-8341416a83de","Type":"ContainerStarted","Data":"f6cd751853a454e2a4501e409e81a48cda7ad91c4044e22a4fdce2695888b8ad"} Jul 14 11:19:35.213071 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.213056 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mq2gr" Jul 14 11:19:35.220457 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.220427 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5j9x" event={"ID":"6a2a6105-4b48-40b0-8f75-2023d94157fa","Type":"ContainerStarted","Data":"ad8a3f71a41d571d707944ae41e72ab9c11a2cb7ac81136fd7d12dff39c37062"} Jul 14 11:19:35.230858 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.230744 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-l8hd4" podStartSLOduration=44.171284131 podStartE2EDuration="47.230728375s" totalImagesPullingTime="3.059444244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:35.229230653 +0000 UTC m=+105.085064825" watchObservedRunningTime="2026-07-14 11:19:35.230728375 +0000 UTC m=+105.086562547" Jul 14 11:19:35.251632 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.251579 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z5j9x" podStartSLOduration=33.737671244 podStartE2EDuration="42.251559436s" totalImagesPullingTime="8.513888192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:35.250438611 +0000 UTC m=+105.106272786" watchObservedRunningTime="2026-07-14 11:19:35.251559436 +0000 UTC m=+105.107393608" Jul 14 11:19:35.276482 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.276432 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mq2gr" podStartSLOduration=64.766393923 podStartE2EDuration="1m8.276416922s" totalImagesPullingTime="3.510022999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:35.275006033 +0000 UTC m=+105.130840206" watchObservedRunningTime="2026-07-14 11:19:35.276416922 +0000 UTC m=+105.132251093" Jul 14 11:19:35.617636 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:35.617601 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-l8hd4" Jul 14 11:19:36.226141 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:36.226111 2568 generic.go:356] "Generic (PLEG): container finished" podID="4464b705-8365-4a4f-ba5b-935b8c09e6ba" containerID="79cac9139acadbafcba7fec71d217dea3c9b1ef33f20c61953edf6cf6a9d4292" exitCode=0 Jul 14 11:19:36.226613 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:36.226225 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k7r8r" event={"ID":"4464b705-8365-4a4f-ba5b-935b8c09e6ba","Type":"ContainerDied","Data":"79cac9139acadbafcba7fec71d217dea3c9b1ef33f20c61953edf6cf6a9d4292"} Jul 14 11:19:37.233596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:37.233548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k7r8r" event={"ID":"4464b705-8365-4a4f-ba5b-935b8c09e6ba","Type":"ContainerStarted","Data":"5bf933fde24b76d88ad760cc39ee41f63fe45d542398db2d9cf0995cae4cf768"} Jul 14 11:19:37.234039 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:37.233631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k7r8r" event={"ID":"4464b705-8365-4a4f-ba5b-935b8c09e6ba","Type":"ContainerStarted","Data":"3b6eb304f0f2c75ac97c21011b8dd89a50a501de1d0bfa137ffcc6d61319091c"} Jul 14 11:19:37.258983 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:37.258939 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k7r8r" podStartSLOduration=4.29450777 podStartE2EDuration="5.258921041s" totalImagesPullingTime="888.469505ms" totalInitContainerRuntime="75.943766ms" isStatefulPod=true podCreationTimestamp="2026-07-14 11:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:37.256674323 +0000 UTC m=+107.112508507" watchObservedRunningTime="2026-07-14 11:19:37.258921041 +0000 UTC m=+107.114755211" Jul 14 11:19:44.119663 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:44.119627 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-678db546b5-v268x"] Jul 14 11:19:45.229098 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:45.229069 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mq2gr" Jul 14 11:19:54.124976 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:19:54.124944 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:20:00.600196 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:00.600155 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:20:00.602564 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:00.602544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecc55306-170e-463f-b3e5-263e3201fc1c-metrics-certs\") pod \"network-metrics-daemon-8dclj\" (UID: \"ecc55306-170e-463f-b3e5-263e3201fc1c\") " pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:20:00.684875 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:00.684849 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bm9fn\"" Jul 14 11:20:00.692563 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:00.692544 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dclj" Jul 14 11:20:00.821016 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:00.820992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8dclj"] Jul 14 11:20:00.823561 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:20:00.823538 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc55306_170e_463f_b3e5_263e3201fc1c.slice/crio-9106aaad9fdd6813d139d8ed98f31534f6ae59986eea16daaea4db41577d08dd WatchSource:0}: Error finding container 9106aaad9fdd6813d139d8ed98f31534f6ae59986eea16daaea4db41577d08dd: Status 404 returned error can't find the container with id 9106aaad9fdd6813d139d8ed98f31534f6ae59986eea16daaea4db41577d08dd Jul 14 11:20:01.324506 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:01.324471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dclj" event={"ID":"ecc55306-170e-463f-b3e5-263e3201fc1c","Type":"ContainerStarted","Data":"9106aaad9fdd6813d139d8ed98f31534f6ae59986eea16daaea4db41577d08dd"} Jul 14 11:20:03.333726 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:03.333685 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dclj" event={"ID":"ecc55306-170e-463f-b3e5-263e3201fc1c","Type":"ContainerStarted","Data":"b44643171844172b6bf1d2d1ad8376c7ffceb0c15deb35f5479a8aa3ea9ca61e"} Jul 14 11:20:03.334184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:03.333751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dclj" event={"ID":"ecc55306-170e-463f-b3e5-263e3201fc1c","Type":"ContainerStarted","Data":"6aa699e9f309193b712e57068f20c8cb2e0988e9b3ced2ed3e93eb0be0c789a0"} Jul 14 11:20:03.335186 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:03.335164 2568 generic.go:356] "Generic (PLEG): container finished" podID="8ca8083f-bc69-461c-80e3-d5abc98eb611" containerID="e1409590465b5e3d29adbb6f67336300538ca6b21e9ba8c78c41964d88a21863" exitCode=0 Jul 14 11:20:03.335291 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:03.335203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" event={"ID":"8ca8083f-bc69-461c-80e3-d5abc98eb611","Type":"ContainerDied","Data":"e1409590465b5e3d29adbb6f67336300538ca6b21e9ba8c78c41964d88a21863"} Jul 14 11:20:03.335492 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:03.335474 2568 scope.go:117] "RemoveContainer" containerID="e1409590465b5e3d29adbb6f67336300538ca6b21e9ba8c78c41964d88a21863" Jul 14 11:20:03.400573 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:03.400498 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8dclj" podStartSLOduration=130.934864279 podStartE2EDuration="2m12.400482585s" totalImagesPullingTime="1.465618306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:20:03.367776158 +0000 UTC m=+133.223610326" watchObservedRunningTime="2026-07-14 11:20:03.400482585 +0000 UTC m=+133.256316755" Jul 14 11:20:04.340964 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:04.340929 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wl2m9" event={"ID":"8ca8083f-bc69-461c-80e3-d5abc98eb611","Type":"ContainerStarted","Data":"df0ad6d25dd9442ac47760a2575a30b949d33e4261572e8de6b362e26617d6fd"} Jul 14 11:20:06.414754 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:06.414726 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-rjc8c_17070e67-dd32-4d46-a9c1-7e2d634e2ccd/cluster-monitoring-operator/0.log" Jul 14 11:20:07.349829 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:07.349768 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" podUID="5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 14 11:20:07.611613 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:07.611533 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k7r8r_4464b705-8365-4a4f-ba5b-935b8c09e6ba/init-textfile/0.log" Jul 14 11:20:07.813160 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:07.813125 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k7r8r_4464b705-8365-4a4f-ba5b-935b8c09e6ba/node-exporter/0.log" Jul 14 11:20:08.012392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:08.012311 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k7r8r_4464b705-8365-4a4f-ba5b-935b8c09e6ba/kube-rbac-proxy/0.log" Jul 14 11:20:09.140070 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.140019 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-678db546b5-v268x" podUID="ed757a72-785a-4619-b72d-aa643cf0ae32" containerName="registry" containerID="cri-o://8b119ad3163a5b1c4d2f1901a2a1456e73aeaf36a8aaf1bfbaed3b52a0b23896" gracePeriod=30 Jul 14 11:20:09.361335 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.361304 2568 generic.go:356] "Generic (PLEG): container finished" podID="ed757a72-785a-4619-b72d-aa643cf0ae32" containerID="8b119ad3163a5b1c4d2f1901a2a1456e73aeaf36a8aaf1bfbaed3b52a0b23896" exitCode=0 Jul 14 11:20:09.361485 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.361384 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678db546b5-v268x" event={"ID":"ed757a72-785a-4619-b72d-aa643cf0ae32","Type":"ContainerDied","Data":"8b119ad3163a5b1c4d2f1901a2a1456e73aeaf36a8aaf1bfbaed3b52a0b23896"} Jul 14 11:20:09.425323 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.425294 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:20:09.468609 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468578 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-bound-sa-token\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.468609 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468610 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-certificates\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.468837 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468636 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.468837 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468674 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-image-registry-private-configuration\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.468837 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468713 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed757a72-785a-4619-b72d-aa643cf0ae32-ca-trust-extracted\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.468837 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468783 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-trusted-ca\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.468837 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468817 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5b65\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-kube-api-access-c5b65\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.469065 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.468867 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-installation-pull-secrets\") pod \"ed757a72-785a-4619-b72d-aa643cf0ae32\" (UID: \"ed757a72-785a-4619-b72d-aa643cf0ae32\") " Jul 14 11:20:09.469177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.469046 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:20:09.469529 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.469486 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:20:09.472037 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.471822 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:20:09.472037 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.471987 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:20:09.472037 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.471996 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:20:09.472200 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.472080 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-kube-api-access-c5b65" (OuterVolumeSpecName: "kube-api-access-c5b65") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "kube-api-access-c5b65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:20:09.472237 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.472211 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:20:09.477106 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.477075 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed757a72-785a-4619-b72d-aa643cf0ae32-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ed757a72-785a-4619-b72d-aa643cf0ae32" (UID: "ed757a72-785a-4619-b72d-aa643cf0ae32"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:20:09.569590 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569555 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-installation-pull-secrets\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569590 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569583 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-bound-sa-token\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569590 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569596 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-certificates\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569819 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569608 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-registry-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569819 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569619 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed757a72-785a-4619-b72d-aa643cf0ae32-image-registry-private-configuration\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569819 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569631 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed757a72-785a-4619-b72d-aa643cf0ae32-ca-trust-extracted\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569819 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569641 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed757a72-785a-4619-b72d-aa643cf0ae32-trusted-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:09.569819 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:09.569652 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5b65\" (UniqueName: \"kubernetes.io/projected/ed757a72-785a-4619-b72d-aa643cf0ae32-kube-api-access-c5b65\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:20:10.370706 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:10.370672 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-678db546b5-v268x" event={"ID":"ed757a72-785a-4619-b72d-aa643cf0ae32","Type":"ContainerDied","Data":"29ed3aa63fc4af0a8b56fa9012b39d26a8f7196ffbf0ef7f7f04276e4a31049e"} Jul 14 11:20:10.371184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:10.370747 2568 scope.go:117] "RemoveContainer" containerID="8b119ad3163a5b1c4d2f1901a2a1456e73aeaf36a8aaf1bfbaed3b52a0b23896" Jul 14 11:20:10.371184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:10.370683 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-678db546b5-v268x" Jul 14 11:20:10.392669 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:10.392648 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-678db546b5-v268x"] Jul 14 11:20:10.396665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:10.396645 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-678db546b5-v268x"] Jul 14 11:20:10.676633 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:10.676555 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed757a72-785a-4619-b72d-aa643cf0ae32" path="/var/lib/kubelet/pods/ed757a72-785a-4619-b72d-aa643cf0ae32/volumes" Jul 14 11:20:13.412315 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:13.412287 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:20:13.614320 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:13.614275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/3.log" Jul 14 11:20:14.412990 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:14.412959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794fd9f96d-dt6lg_00628d94-4a03-456d-8c41-0ab813236ad1/router/0.log" Jul 14 11:20:14.810796 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:14.810765 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9jdh_648326eb-7168-4bdb-a3e0-e7b510948491/serve-healthcheck-canary/0.log" Jul 14 11:20:16.396240 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:16.396209 2568 generic.go:356] "Generic (PLEG): container finished" podID="c6eb804d-f51c-45b4-a39d-e56acbe6d53f" containerID="356d4521a1287e2970d7e4bdd50a0022488cbc73d91ab85c5ac874c6be0242c0" exitCode=0 Jul 14 11:20:16.396641 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:16.396324 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" event={"ID":"c6eb804d-f51c-45b4-a39d-e56acbe6d53f","Type":"ContainerDied","Data":"356d4521a1287e2970d7e4bdd50a0022488cbc73d91ab85c5ac874c6be0242c0"} Jul 14 11:20:16.396641 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:16.396616 2568 scope.go:117] "RemoveContainer" containerID="356d4521a1287e2970d7e4bdd50a0022488cbc73d91ab85c5ac874c6be0242c0" Jul 14 11:20:17.350444 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:17.350405 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" podUID="5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 14 11:20:17.402689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:17.402645 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-l2cns" event={"ID":"c6eb804d-f51c-45b4-a39d-e56acbe6d53f","Type":"ContainerStarted","Data":"432bce535bf902f2dd53a6e7637c96deb39ac1bc61414a4b2cf5e2e0831e4750"} Jul 14 11:20:19.418217 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:19.418184 2568 generic.go:356] "Generic (PLEG): container finished" podID="2163c957-e628-40c6-b7ca-57601411acfe" containerID="f1048c140124d71184fa1c138d0a53e59c7811db9add70e6e6e2b1c9121ea056" exitCode=0 Jul 14 11:20:19.418605 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:19.418245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" event={"ID":"2163c957-e628-40c6-b7ca-57601411acfe","Type":"ContainerDied","Data":"f1048c140124d71184fa1c138d0a53e59c7811db9add70e6e6e2b1c9121ea056"} Jul 14 11:20:19.418605 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:19.418601 2568 scope.go:117] "RemoveContainer" containerID="f1048c140124d71184fa1c138d0a53e59c7811db9add70e6e6e2b1c9121ea056" Jul 14 11:20:20.424188 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:20.424154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42z4g" event={"ID":"2163c957-e628-40c6-b7ca-57601411acfe","Type":"ContainerStarted","Data":"e04cd94be02179a9ec1740531bd16d525a5df6d8948c1145abf46082b467a82f"} Jul 14 11:20:27.349548 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:27.349501 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" podUID="5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 14 11:20:27.350034 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:27.349577 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" Jul 14 11:20:27.350197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:27.350162 2568 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"8fe01f04887d2a753d3bc5ce2250d555d4a0a4409c4eae4e9ae93d8d86da7cf4"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 14 11:20:27.350294 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:27.350225 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" podUID="5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b" containerName="service-proxy" containerID="cri-o://8fe01f04887d2a753d3bc5ce2250d555d4a0a4409c4eae4e9ae93d8d86da7cf4" gracePeriod=30 Jul 14 11:20:28.456170 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:28.456139 2568 generic.go:356] "Generic (PLEG): container finished" podID="5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b" containerID="8fe01f04887d2a753d3bc5ce2250d555d4a0a4409c4eae4e9ae93d8d86da7cf4" exitCode=2 Jul 14 11:20:28.456540 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:28.456200 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" event={"ID":"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b","Type":"ContainerDied","Data":"8fe01f04887d2a753d3bc5ce2250d555d4a0a4409c4eae4e9ae93d8d86da7cf4"} Jul 14 11:20:28.456540 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:20:28.456235 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775b664c7-bml4c" event={"ID":"5ec5b0ad-926a-4d10-8d21-88e0c6bcee4b","Type":"ContainerStarted","Data":"4fb7cd9c960c2edbb9c751f8385b64c5aee0e81e5889f09d8e47764b751843ee"} Jul 14 11:22:50.559486 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:22:50.559455 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:22:50.563058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:22:50.563036 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:22:50.566990 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:22:50.566971 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:22:50.571239 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:22:50.571217 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 11:22:50.571341 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:22:50.571321 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:24:24.584913 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.584837 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd"] Jul 14 11:24:24.585410 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.585106 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed757a72-785a-4619-b72d-aa643cf0ae32" containerName="registry" Jul 14 11:24:24.585410 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.585126 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed757a72-785a-4619-b72d-aa643cf0ae32" containerName="registry" Jul 14 11:24:24.585410 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.585191 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed757a72-785a-4619-b72d-aa643cf0ae32" containerName="registry" Jul 14 11:24:24.588072 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.588057 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.591694 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.591672 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 11:24:24.591830 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.591815 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 11:24:24.591936 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.591920 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gz7sx\"" Jul 14 11:24:24.591983 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.591945 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 11:24:24.592034 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.592001 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 11:24:24.604826 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.604804 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd"] Jul 14 11:24:24.770983 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.770942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.771166 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.770992 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qtsj\" (UniqueName: \"kubernetes.io/projected/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-kube-api-access-4qtsj\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.771166 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.771050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.771166 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.771095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.771166 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.771121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.871997 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.871911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.871997 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.871959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qtsj\" (UniqueName: \"kubernetes.io/projected/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-kube-api-access-4qtsj\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.872201 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.872075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.872201 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.872129 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.872201 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.872151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.872510 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.872487 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.874514 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.874447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.874707 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.874666 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.874787 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.874728 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.883731 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.883704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qtsj\" (UniqueName: \"kubernetes.io/projected/e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1-kube-api-access-4qtsj\") pod \"opendatahub-operator-controller-manager-555b6fb65-zwpqd\" (UID: \"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:24.898362 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:24.898343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:25.038950 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:25.038901 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd"] Jul 14 11:24:25.041931 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:24:25.041903 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19a6ce3_faf7_4a3d_bdbf_0c2008f3b5c1.slice/crio-f30bacd747cb9c565bf4d409032edb68c48418ca8c96c4e1afed9fa751f30c22 WatchSource:0}: Error finding container f30bacd747cb9c565bf4d409032edb68c48418ca8c96c4e1afed9fa751f30c22: Status 404 returned error can't find the container with id f30bacd747cb9c565bf4d409032edb68c48418ca8c96c4e1afed9fa751f30c22 Jul 14 11:24:25.043666 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:25.043651 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:24:25.324656 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:25.324620 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" event={"ID":"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1","Type":"ContainerStarted","Data":"f30bacd747cb9c565bf4d409032edb68c48418ca8c96c4e1afed9fa751f30c22"} Jul 14 11:24:28.342451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:28.342419 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" event={"ID":"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1","Type":"ContainerStarted","Data":"1df87c33b7cef64df26020d117cc5fa0df30ba905c3d742b76d540324a79e3de"} Jul 14 11:24:29.347853 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:29.347817 2568 generic.go:356] "Generic (PLEG): container finished" podID="e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1" containerID="1df87c33b7cef64df26020d117cc5fa0df30ba905c3d742b76d540324a79e3de" exitCode=0 Jul 14 11:24:29.348251 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:29.347896 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" event={"ID":"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1","Type":"ContainerDied","Data":"1df87c33b7cef64df26020d117cc5fa0df30ba905c3d742b76d540324a79e3de"} Jul 14 11:24:30.355738 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:30.355701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" event={"ID":"e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1","Type":"ContainerStarted","Data":"4de59c0c94d162f51bb1484eba0e160f0e59cb9dac8ed3e5c854c8f859b5bbc4"} Jul 14 11:24:30.356154 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:30.355851 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:30.381954 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:30.381890 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" podStartSLOduration=1.9885141819999999 podStartE2EDuration="6.381875628s" totalImagesPullingTime="3.323483943s" totalInitContainerRuntime="1.069877503s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:30.38112714 +0000 UTC m=+400.236961297" watchObservedRunningTime="2026-07-14 11:24:30.381875628 +0000 UTC m=+400.237709799" Jul 14 11:24:41.362332 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:41.362303 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-zwpqd" Jul 14 11:24:44.781793 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.781759 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc"] Jul 14 11:24:44.785057 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.785040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.788045 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.788025 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 11:24:44.788045 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.788032 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 11:24:44.804340 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.804309 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc"] Jul 14 11:24:44.823073 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.823039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa33eac3-211c-4b2f-a40b-04564300efd9-tls-certs\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.823231 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.823086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa33eac3-211c-4b2f-a40b-04564300efd9-tmp\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.823231 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.823129 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p86kb\" (UniqueName: \"kubernetes.io/projected/fa33eac3-211c-4b2f-a40b-04564300efd9-kube-api-access-p86kb\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.923849 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.923804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa33eac3-211c-4b2f-a40b-04564300efd9-tls-certs\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.923849 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.923849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa33eac3-211c-4b2f-a40b-04564300efd9-tmp\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.924098 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.923888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p86kb\" (UniqueName: \"kubernetes.io/projected/fa33eac3-211c-4b2f-a40b-04564300efd9-kube-api-access-p86kb\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.924098 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:44.923985 2568 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 11:24:44.924098 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:44.924065 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa33eac3-211c-4b2f-a40b-04564300efd9-tls-certs podName:fa33eac3-211c-4b2f-a40b-04564300efd9 nodeName:}" failed. No retries permitted until 2026-07-14 11:24:45.424045089 +0000 UTC m=+415.279879243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fa33eac3-211c-4b2f-a40b-04564300efd9-tls-certs") pod "kube-auth-proxy-76f9775b55-9b5zc" (UID: "fa33eac3-211c-4b2f-a40b-04564300efd9") : secret "kube-auth-proxy-tls" not found Jul 14 11:24:44.926421 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.926396 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa33eac3-211c-4b2f-a40b-04564300efd9-tmp\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.945787 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.945748 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p86kb\" (UniqueName: \"kubernetes.io/projected/fa33eac3-211c-4b2f-a40b-04564300efd9-kube-api-access-p86kb\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:44.999552 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:44.999518 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d2jfx"] Jul 14 11:24:45.002747 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.002723 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.006264 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.006235 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 14 11:24:45.006373 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.006277 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6tr4g\"" Jul 14 11:24:45.024594 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.024565 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08aba400-78d1-44be-8921-7112772a70aa-cert\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.024766 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.024619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d85h6\" (UniqueName: \"kubernetes.io/projected/08aba400-78d1-44be-8921-7112772a70aa-kube-api-access-d85h6\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.045994 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.045901 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d2jfx"] Jul 14 11:24:45.125755 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.125712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08aba400-78d1-44be-8921-7112772a70aa-cert\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.125930 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.125773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d85h6\" (UniqueName: \"kubernetes.io/projected/08aba400-78d1-44be-8921-7112772a70aa-kube-api-access-d85h6\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.125930 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:45.125891 2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 14 11:24:45.125996 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:45.125977 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08aba400-78d1-44be-8921-7112772a70aa-cert podName:08aba400-78d1-44be-8921-7112772a70aa nodeName:}" failed. No retries permitted until 2026-07-14 11:24:45.625950526 +0000 UTC m=+415.481784675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/08aba400-78d1-44be-8921-7112772a70aa-cert") pod "odh-model-controller-7998c67dc4-d2jfx" (UID: "08aba400-78d1-44be-8921-7112772a70aa") : secret "odh-model-controller-webhook-cert" not found Jul 14 11:24:45.139736 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.139704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d85h6\" (UniqueName: \"kubernetes.io/projected/08aba400-78d1-44be-8921-7112772a70aa-kube-api-access-d85h6\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.428759 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.428724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa33eac3-211c-4b2f-a40b-04564300efd9-tls-certs\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:45.431411 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.431385 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa33eac3-211c-4b2f-a40b-04564300efd9-tls-certs\") pod \"kube-auth-proxy-76f9775b55-9b5zc\" (UID: \"fa33eac3-211c-4b2f-a40b-04564300efd9\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:45.630576 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.630536 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08aba400-78d1-44be-8921-7112772a70aa-cert\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.633154 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.633119 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08aba400-78d1-44be-8921-7112772a70aa-cert\") pod \"odh-model-controller-7998c67dc4-d2jfx\" (UID: \"08aba400-78d1-44be-8921-7112772a70aa\") " pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.694106 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.694021 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" Jul 14 11:24:45.868428 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.868384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc"] Jul 14 11:24:45.872373 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:24:45.872341 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa33eac3_211c_4b2f_a40b_04564300efd9.slice/crio-8dc5262953c75d546bad5b3e1c4fe12f3d79f8f928785f09d09b1f8013e75cc6 WatchSource:0}: Error finding container 8dc5262953c75d546bad5b3e1c4fe12f3d79f8f928785f09d09b1f8013e75cc6: Status 404 returned error can't find the container with id 8dc5262953c75d546bad5b3e1c4fe12f3d79f8f928785f09d09b1f8013e75cc6 Jul 14 11:24:45.885490 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.885461 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fsrww"] Jul 14 11:24:45.889732 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.889711 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:45.896083 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.896061 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 11:24:45.896083 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.896071 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-mbmcr\"" Jul 14 11:24:45.915506 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.915472 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:45.927001 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.926949 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fsrww"] Jul 14 11:24:45.933923 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.933877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq65h\" (UniqueName: \"kubernetes.io/projected/278a93c6-7346-469e-a8e2-b5f7abc77f11-kube-api-access-mq65h\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:45.934097 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:45.934017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278a93c6-7346-469e-a8e2-b5f7abc77f11-tls-certs\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.034918 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.034870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq65h\" (UniqueName: \"kubernetes.io/projected/278a93c6-7346-469e-a8e2-b5f7abc77f11-kube-api-access-mq65h\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.035064 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.034962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278a93c6-7346-469e-a8e2-b5f7abc77f11-tls-certs\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.035137 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:46.035118 2568 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 14 11:24:46.035206 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:46.035194 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/278a93c6-7346-469e-a8e2-b5f7abc77f11-tls-certs podName:278a93c6-7346-469e-a8e2-b5f7abc77f11 nodeName:}" failed. No retries permitted until 2026-07-14 11:24:46.535169903 +0000 UTC m=+416.391004079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/278a93c6-7346-469e-a8e2-b5f7abc77f11-tls-certs") pod "model-serving-api-54d89bd5fc-fsrww" (UID: "278a93c6-7346-469e-a8e2-b5f7abc77f11") : secret "model-serving-api-tls" not found Jul 14 11:24:46.051362 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.051323 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq65h\" (UniqueName: \"kubernetes.io/projected/278a93c6-7346-469e-a8e2-b5f7abc77f11-kube-api-access-mq65h\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.071809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.071784 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d2jfx"] Jul 14 11:24:46.074005 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:24:46.073979 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08aba400_78d1_44be_8921_7112772a70aa.slice/crio-2f5b0d7b0e94f2bb101bdf83a5a4c444bea5bb712093d6345a7d16c9407aa476 WatchSource:0}: Error finding container 2f5b0d7b0e94f2bb101bdf83a5a4c444bea5bb712093d6345a7d16c9407aa476: Status 404 returned error can't find the container with id 2f5b0d7b0e94f2bb101bdf83a5a4c444bea5bb712093d6345a7d16c9407aa476 Jul 14 11:24:46.426349 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.426299 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" event={"ID":"08aba400-78d1-44be-8921-7112772a70aa","Type":"ContainerStarted","Data":"2f5b0d7b0e94f2bb101bdf83a5a4c444bea5bb712093d6345a7d16c9407aa476"} Jul 14 11:24:46.427596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.427566 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" event={"ID":"fa33eac3-211c-4b2f-a40b-04564300efd9","Type":"ContainerStarted","Data":"8dc5262953c75d546bad5b3e1c4fe12f3d79f8f928785f09d09b1f8013e75cc6"} Jul 14 11:24:46.539546 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.539511 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278a93c6-7346-469e-a8e2-b5f7abc77f11-tls-certs\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.542077 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.542050 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/278a93c6-7346-469e-a8e2-b5f7abc77f11-tls-certs\") pod \"model-serving-api-54d89bd5fc-fsrww\" (UID: \"278a93c6-7346-469e-a8e2-b5f7abc77f11\") " pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.610522 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.610478 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh"] Jul 14 11:24:46.613897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.613870 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.616869 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.616838 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 11:24:46.617080 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.617037 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b94mv\"" Jul 14 11:24:46.617215 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.617183 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 11:24:46.617319 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.617309 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 11:24:46.617383 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.617374 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 11:24:46.632278 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.632233 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh"] Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.640957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-token\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.641011 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.641060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.641091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd470fd2-7f4a-47a7-a958-2ae6833453f5-local-certs\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.641140 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-cacerts\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.641185 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.641385 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.641221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4k8t\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-kube-api-access-j4k8t\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742532 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742445 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd470fd2-7f4a-47a7-a958-2ae6833453f5-local-certs\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742532 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742503 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-cacerts\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4k8t\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-kube-api-access-j4k8t\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742627 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-token\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.742763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.742697 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.743584 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.743532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.746501 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.746456 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-cacerts\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.747135 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.747111 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.747473 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.747453 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.747656 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.747631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd470fd2-7f4a-47a7-a958-2ae6833453f5-local-certs\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.756539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.756485 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-token\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.757148 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.757052 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4k8t\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-kube-api-access-j4k8t\") pod \"istiod-openshift-gateway-7574459656-2qnxh\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:46.800692 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.800658 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:46.928755 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:46.928718 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:47.006339 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:47.006235 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fsrww"] Jul 14 11:24:47.145327 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:47.145287 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh"] Jul 14 11:24:47.148751 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:24:47.148717 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd470fd2_7f4a_47a7_a958_2ae6833453f5.slice/crio-88d384abb5b09c9bae1168e0808dd7b60f8a834c51c66c5d70bdaa6841ed4592 WatchSource:0}: Error finding container 88d384abb5b09c9bae1168e0808dd7b60f8a834c51c66c5d70bdaa6841ed4592: Status 404 returned error can't find the container with id 88d384abb5b09c9bae1168e0808dd7b60f8a834c51c66c5d70bdaa6841ed4592 Jul 14 11:24:47.436720 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:47.436681 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" event={"ID":"fd470fd2-7f4a-47a7-a958-2ae6833453f5","Type":"ContainerStarted","Data":"88d384abb5b09c9bae1168e0808dd7b60f8a834c51c66c5d70bdaa6841ed4592"} Jul 14 11:24:47.438415 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:47.438352 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" event={"ID":"278a93c6-7346-469e-a8e2-b5f7abc77f11","Type":"ContainerStarted","Data":"ace6e2e1a9316647a98f9d7f3fe79b9cd20898270d522c3355a7f91f353ac3cf"} Jul 14 11:24:51.461219 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.461183 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" event={"ID":"278a93c6-7346-469e-a8e2-b5f7abc77f11","Type":"ContainerStarted","Data":"92309aa4dd2a0d7380b9cef2aaa00b05ffdcf72dfb8204a0dcfcdd669924c467"} Jul 14 11:24:51.461731 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.461324 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:24:51.462969 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.462944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" event={"ID":"08aba400-78d1-44be-8921-7112772a70aa","Type":"ContainerStarted","Data":"3bec02263d9b589fd50e19f9ae228828aa7a2075e593a931685f09e4dc57aef8"} Jul 14 11:24:51.463056 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.463001 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:24:51.464613 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.464583 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" event={"ID":"fa33eac3-211c-4b2f-a40b-04564300efd9","Type":"ContainerStarted","Data":"a890cc2724cb87b090227b2b51f4ecde79cac4ab7b69a7a92ceb394c9bbb5874"} Jul 14 11:24:51.479412 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.479353 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" podStartSLOduration=3.021129091 podStartE2EDuration="6.47933365s" totalImagesPullingTime="3.458204559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:51.477455153 +0000 UTC m=+421.333289326" watchObservedRunningTime="2026-07-14 11:24:51.47933365 +0000 UTC m=+421.335167824" Jul 14 11:24:51.519516 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.519444 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76f9775b55-9b5zc" podStartSLOduration=2.9184578180000003 podStartE2EDuration="7.519424047s" totalImagesPullingTime="4.600966229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:51.496963317 +0000 UTC m=+421.352797488" watchObservedRunningTime="2026-07-14 11:24:51.519424047 +0000 UTC m=+421.375258220" Jul 14 11:24:51.520337 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.520290 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" podStartSLOduration=3.123711022 podStartE2EDuration="7.52027797s" totalImagesPullingTime="4.396566948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:51.519704803 +0000 UTC m=+421.375538975" watchObservedRunningTime="2026-07-14 11:24:51.52027797 +0000 UTC m=+421.376112133" Jul 14 11:24:51.904290 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.904228 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 14 11:24:51.904402 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:51.904335 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 14 11:24:52.470755 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:52.470712 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" event={"ID":"fd470fd2-7f4a-47a7-a958-2ae6833453f5","Type":"ContainerStarted","Data":"8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391"} Jul 14 11:24:52.471217 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:52.470848 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:52.472496 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:52.472468 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2qnxh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:24:52.472631 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:52.472521 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:24:52.496380 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:52.496326 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" podStartSLOduration=1.744219892 podStartE2EDuration="6.496311837s" totalImagesPullingTime="4.752091945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:52.494247573 +0000 UTC m=+422.350081745" watchObservedRunningTime="2026-07-14 11:24:52.496311837 +0000 UTC m=+422.352146009" Jul 14 11:24:53.476030 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:53.475989 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2qnxh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:24:53.476506 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:53.476089 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:24:56.477041 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:56.477008 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:24:57.038113 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.038074 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc"] Jul 14 11:24:57.044545 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.044510 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.052592 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.052562 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 14 11:24:57.052751 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.052572 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vdhrh\"" Jul 14 11:24:57.059551 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.059523 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9"] Jul 14 11:24:57.063776 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.063745 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.067512 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.067487 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-52zz9\"" Jul 14 11:24:57.067612 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.067536 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 14 11:24:57.076192 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.076168 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc"] Jul 14 11:24:57.094049 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.094019 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9"] Jul 14 11:24:57.140431 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.140399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76879cef-26ef-4c81-adef-9fa7a7fa7a43-cert\") pod \"kserve-controller-manager-d6f4b65b8-vp8rc\" (UID: \"76879cef-26ef-4c81-adef-9fa7a7fa7a43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.140431 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.140437 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcprt\" (UniqueName: \"kubernetes.io/projected/94c6e106-e6c7-47ea-8682-f1c54fc2d665-kube-api-access-wcprt\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.140680 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.140536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxx6l\" (UniqueName: \"kubernetes.io/projected/76879cef-26ef-4c81-adef-9fa7a7fa7a43-kube-api-access-bxx6l\") pod \"kserve-controller-manager-d6f4b65b8-vp8rc\" (UID: \"76879cef-26ef-4c81-adef-9fa7a7fa7a43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.140680 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.140568 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c6e106-e6c7-47ea-8682-f1c54fc2d665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.241980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.241943 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76879cef-26ef-4c81-adef-9fa7a7fa7a43-cert\") pod \"kserve-controller-manager-d6f4b65b8-vp8rc\" (UID: \"76879cef-26ef-4c81-adef-9fa7a7fa7a43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.241980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.241985 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcprt\" (UniqueName: \"kubernetes.io/projected/94c6e106-e6c7-47ea-8682-f1c54fc2d665-kube-api-access-wcprt\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.242178 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.242041 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxx6l\" (UniqueName: \"kubernetes.io/projected/76879cef-26ef-4c81-adef-9fa7a7fa7a43-kube-api-access-bxx6l\") pod \"kserve-controller-manager-d6f4b65b8-vp8rc\" (UID: \"76879cef-26ef-4c81-adef-9fa7a7fa7a43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.242178 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.242059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c6e106-e6c7-47ea-8682-f1c54fc2d665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.242178 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:57.242154 2568 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 14 11:24:57.242302 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:24:57.242219 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c6e106-e6c7-47ea-8682-f1c54fc2d665-cert podName:94c6e106-e6c7-47ea-8682-f1c54fc2d665 nodeName:}" failed. No retries permitted until 2026-07-14 11:24:57.742198093 +0000 UTC m=+427.598032243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c6e106-e6c7-47ea-8682-f1c54fc2d665-cert") pod "llmisvc-controller-manager-777c6bd7d5-q5dc9" (UID: "94c6e106-e6c7-47ea-8682-f1c54fc2d665") : secret "llmisvc-webhook-server-cert" not found Jul 14 11:24:57.244572 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.244547 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76879cef-26ef-4c81-adef-9fa7a7fa7a43-cert\") pod \"kserve-controller-manager-d6f4b65b8-vp8rc\" (UID: \"76879cef-26ef-4c81-adef-9fa7a7fa7a43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.255884 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.255852 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxx6l\" (UniqueName: \"kubernetes.io/projected/76879cef-26ef-4c81-adef-9fa7a7fa7a43-kube-api-access-bxx6l\") pod \"kserve-controller-manager-d6f4b65b8-vp8rc\" (UID: \"76879cef-26ef-4c81-adef-9fa7a7fa7a43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.255994 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.255858 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcprt\" (UniqueName: \"kubernetes.io/projected/94c6e106-e6c7-47ea-8682-f1c54fc2d665-kube-api-access-wcprt\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.359669 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.359633 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:24:57.519532 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.519458 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc"] Jul 14 11:24:57.522207 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:24:57.522181 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76879cef_26ef_4c81_adef_9fa7a7fa7a43.slice/crio-63694b41cae5d78f3a62f2557987a11de5f9e0effca1e1cfb6c9856e93f94649 WatchSource:0}: Error finding container 63694b41cae5d78f3a62f2557987a11de5f9e0effca1e1cfb6c9856e93f94649: Status 404 returned error can't find the container with id 63694b41cae5d78f3a62f2557987a11de5f9e0effca1e1cfb6c9856e93f94649 Jul 14 11:24:57.745787 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.745693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c6e106-e6c7-47ea-8682-f1c54fc2d665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.748112 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.748086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c6e106-e6c7-47ea-8682-f1c54fc2d665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q5dc9\" (UID: \"94c6e106-e6c7-47ea-8682-f1c54fc2d665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:57.975138 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:57.975101 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:24:58.134012 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:58.133971 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9"] Jul 14 11:24:58.137031 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:24:58.136989 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod94c6e106_e6c7_47ea_8682_f1c54fc2d665.slice/crio-662983b53b8f991be02b95730388ad693ecc3762a660b8280a974c098fe18134 WatchSource:0}: Error finding container 662983b53b8f991be02b95730388ad693ecc3762a660b8280a974c098fe18134: Status 404 returned error can't find the container with id 662983b53b8f991be02b95730388ad693ecc3762a660b8280a974c098fe18134 Jul 14 11:24:58.499763 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:58.499684 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" event={"ID":"94c6e106-e6c7-47ea-8682-f1c54fc2d665","Type":"ContainerStarted","Data":"662983b53b8f991be02b95730388ad693ecc3762a660b8280a974c098fe18134"} Jul 14 11:24:58.501111 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:58.501078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" event={"ID":"76879cef-26ef-4c81-adef-9fa7a7fa7a43","Type":"ContainerStarted","Data":"63694b41cae5d78f3a62f2557987a11de5f9e0effca1e1cfb6c9856e93f94649"} Jul 14 11:24:59.331907 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:59.331871 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh"] Jul 14 11:24:59.332336 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:59.332102 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" containerID="cri-o://8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391" gracePeriod=30 Jul 14 11:24:59.476496 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:59.476455 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2qnxh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.27:8080/ready\": dial tcp 10.132.0.27:8080: connect: connection refused" start-of-body= Jul 14 11:24:59.476673 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:24:59.476517 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.27:8080/ready\": dial tcp 10.132.0.27:8080: connect: connection refused" Jul 14 11:25:00.218665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.218639 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:25:00.270761 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270727 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-cacerts\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.270916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270777 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-dns-cert\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.270916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270820 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd470fd2-7f4a-47a7-a958-2ae6833453f5-local-certs\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.270916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270870 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-ca-configmap\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.270916 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270913 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-kubeconfig\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.271109 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270950 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4k8t\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-kube-api-access-j4k8t\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.271109 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.270975 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-token\") pod \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\" (UID: \"fd470fd2-7f4a-47a7-a958-2ae6833453f5\") " Jul 14 11:25:00.271415 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.271368 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:25:00.273742 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.273704 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd470fd2-7f4a-47a7-a958-2ae6833453f5-local-certs" (OuterVolumeSpecName: "local-certs") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:25:00.273990 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.273964 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:25:00.274093 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.274004 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:25:00.274093 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.274044 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:25:00.274093 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.274064 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-kube-api-access-j4k8t" (OuterVolumeSpecName: "kube-api-access-j4k8t") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "kube-api-access-j4k8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:25:00.274233 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.274149 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-cacerts" (OuterVolumeSpecName: "cacerts") pod "fd470fd2-7f4a-47a7-a958-2ae6833453f5" (UID: "fd470fd2-7f4a-47a7-a958-2ae6833453f5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372487 2568 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-kubeconfig\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372517 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4k8t\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-kube-api-access-j4k8t\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372527 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-token\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372535 2568 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-cacerts\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372543 2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-dns-cert\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372551 2568 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd470fd2-7f4a-47a7-a958-2ae6833453f5-local-certs\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.372577 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.372559 2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd470fd2-7f4a-47a7-a958-2ae6833453f5-istio-csr-ca-configmap\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:00.513970 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.513929 2568 generic.go:356] "Generic (PLEG): container finished" podID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerID="8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391" exitCode=0 Jul 14 11:25:00.514131 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.513984 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" Jul 14 11:25:00.514131 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.514018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" event={"ID":"fd470fd2-7f4a-47a7-a958-2ae6833453f5","Type":"ContainerDied","Data":"8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391"} Jul 14 11:25:00.514131 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.514072 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh" event={"ID":"fd470fd2-7f4a-47a7-a958-2ae6833453f5","Type":"ContainerDied","Data":"88d384abb5b09c9bae1168e0808dd7b60f8a834c51c66c5d70bdaa6841ed4592"} Jul 14 11:25:00.514131 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.514098 2568 scope.go:117] "RemoveContainer" containerID="8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391" Jul 14 11:25:00.515650 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.515626 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" event={"ID":"76879cef-26ef-4c81-adef-9fa7a7fa7a43","Type":"ContainerStarted","Data":"1d1a28f012482e1ec66449537e0de2e32d6eb7366cee4ef0b11c770bce71ba8a"} Jul 14 11:25:00.515735 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.515674 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:25:00.524485 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.524465 2568 scope.go:117] "RemoveContainer" containerID="8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391" Jul 14 11:25:00.524771 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:25:00.524751 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391\": container with ID starting with 8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391 not found: ID does not exist" containerID="8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391" Jul 14 11:25:00.524835 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.524777 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391"} err="failed to get container status \"8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391\": rpc error: code = NotFound desc = could not find container \"8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391\": container with ID starting with 8ce95f7154bb9c6406531d3fdfcd9aaa12a3f057581488f6ac969ae45b45d391 not found: ID does not exist" Jul 14 11:25:00.567410 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.567350 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" podStartSLOduration=0.977063044 podStartE2EDuration="3.567327927s" totalImagesPullingTime="2.590264883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:25:00.564608788 +0000 UTC m=+430.420442961" watchObservedRunningTime="2026-07-14 11:25:00.567327927 +0000 UTC m=+430.423162103" Jul 14 11:25:00.593488 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.593454 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh"] Jul 14 11:25:00.599294 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.599264 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2qnxh"] Jul 14 11:25:00.680195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:00.680114 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" path="/var/lib/kubelet/pods/fd470fd2-7f4a-47a7-a958-2ae6833453f5/volumes" Jul 14 11:25:01.522111 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:01.522005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" event={"ID":"94c6e106-e6c7-47ea-8682-f1c54fc2d665","Type":"ContainerStarted","Data":"3fb7555ac8509dc3c1c14464f73929090f94f975814661e2b276e3a94e683155"} Jul 14 11:25:01.522111 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:01.522101 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:25:01.547390 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:01.547325 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" podStartSLOduration=1.464909797 podStartE2EDuration="4.547307987s" totalImagesPullingTime="3.08239819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:25:01.545941733 +0000 UTC m=+431.401775958" watchObservedRunningTime="2026-07-14 11:25:01.547307987 +0000 UTC m=+431.403142160" Jul 14 11:25:02.473007 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:02.472974 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-d2jfx" Jul 14 11:25:02.475246 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:02.475219 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fsrww" Jul 14 11:25:31.528810 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:31.528779 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vp8rc" Jul 14 11:25:32.530319 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:32.530290 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q5dc9" Jul 14 11:25:45.400124 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.400088 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n92fj"] Jul 14 11:25:45.400543 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.400444 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" Jul 14 11:25:45.400543 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.400455 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" Jul 14 11:25:45.400543 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.400515 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd470fd2-7f4a-47a7-a958-2ae6833453f5" containerName="discovery" Jul 14 11:25:45.402501 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.402474 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:45.405999 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.405970 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9zkb6\"" Jul 14 11:25:45.406108 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.406089 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 11:25:45.407343 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.407326 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 11:25:45.423366 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.423340 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n92fj"] Jul 14 11:25:45.434937 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.434912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w292\" (UniqueName: \"kubernetes.io/projected/0d5fac36-28e3-4eda-b863-77e472b4ae93-kube-api-access-2w292\") pod \"kuadrant-operator-catalog-n92fj\" (UID: \"0d5fac36-28e3-4eda-b863-77e472b4ae93\") " pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:45.536131 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.536105 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w292\" (UniqueName: \"kubernetes.io/projected/0d5fac36-28e3-4eda-b863-77e472b4ae93-kube-api-access-2w292\") pod \"kuadrant-operator-catalog-n92fj\" (UID: \"0d5fac36-28e3-4eda-b863-77e472b4ae93\") " pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:45.546069 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.546048 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w292\" (UniqueName: \"kubernetes.io/projected/0d5fac36-28e3-4eda-b863-77e472b4ae93-kube-api-access-2w292\") pod \"kuadrant-operator-catalog-n92fj\" (UID: \"0d5fac36-28e3-4eda-b863-77e472b4ae93\") " pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:45.711654 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.711571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:45.757978 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.757938 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n92fj"] Jul 14 11:25:45.845485 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.845370 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n92fj"] Jul 14 11:25:45.847783 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:25:45.847754 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5fac36_28e3_4eda_b863_77e472b4ae93.slice/crio-f4bbb9c61af7030d0787fd0cd55dd8a3c4e23930519ca57bd1a603eec514a53c WatchSource:0}: Error finding container f4bbb9c61af7030d0787fd0cd55dd8a3c4e23930519ca57bd1a603eec514a53c: Status 404 returned error can't find the container with id f4bbb9c61af7030d0787fd0cd55dd8a3c4e23930519ca57bd1a603eec514a53c Jul 14 11:25:45.967778 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.967698 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2d2x4"] Jul 14 11:25:45.970563 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.970547 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:45.979531 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:45.979502 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2d2x4"] Jul 14 11:25:46.040386 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.040354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpng\" (UniqueName: \"kubernetes.io/projected/ba4c0cca-026e-49dc-8530-83ddc5b0addf-kube-api-access-kmpng\") pod \"kuadrant-operator-catalog-2d2x4\" (UID: \"ba4c0cca-026e-49dc-8530-83ddc5b0addf\") " pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:46.141372 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.141339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpng\" (UniqueName: \"kubernetes.io/projected/ba4c0cca-026e-49dc-8530-83ddc5b0addf-kube-api-access-kmpng\") pod \"kuadrant-operator-catalog-2d2x4\" (UID: \"ba4c0cca-026e-49dc-8530-83ddc5b0addf\") " pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:46.151459 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.151429 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpng\" (UniqueName: \"kubernetes.io/projected/ba4c0cca-026e-49dc-8530-83ddc5b0addf-kube-api-access-kmpng\") pod \"kuadrant-operator-catalog-2d2x4\" (UID: \"ba4c0cca-026e-49dc-8530-83ddc5b0addf\") " pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:46.281403 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.281314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:46.429948 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.429923 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2d2x4"] Jul 14 11:25:46.452526 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:25:46.452497 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4c0cca_026e_49dc_8530_83ddc5b0addf.slice/crio-76803a00fcbd1bd7ec9500ae775c88915e34b470a4fcc13d5dffbe3bfa066702 WatchSource:0}: Error finding container 76803a00fcbd1bd7ec9500ae775c88915e34b470a4fcc13d5dffbe3bfa066702: Status 404 returned error can't find the container with id 76803a00fcbd1bd7ec9500ae775c88915e34b470a4fcc13d5dffbe3bfa066702 Jul 14 11:25:46.724075 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.724034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" event={"ID":"ba4c0cca-026e-49dc-8530-83ddc5b0addf","Type":"ContainerStarted","Data":"76803a00fcbd1bd7ec9500ae775c88915e34b470a4fcc13d5dffbe3bfa066702"} Jul 14 11:25:46.725193 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:46.725164 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" event={"ID":"0d5fac36-28e3-4eda-b863-77e472b4ae93","Type":"ContainerStarted","Data":"f4bbb9c61af7030d0787fd0cd55dd8a3c4e23930519ca57bd1a603eec514a53c"} Jul 14 11:25:48.735688 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:48.735647 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" event={"ID":"0d5fac36-28e3-4eda-b863-77e472b4ae93","Type":"ContainerStarted","Data":"2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47"} Jul 14 11:25:48.736140 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:48.735671 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" podUID="0d5fac36-28e3-4eda-b863-77e472b4ae93" containerName="registry-server" containerID="cri-o://2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47" gracePeriod=2 Jul 14 11:25:48.737157 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:48.737125 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" event={"ID":"ba4c0cca-026e-49dc-8530-83ddc5b0addf","Type":"ContainerStarted","Data":"3cab28be9318da0221268838bd9fec476d3cc9ef0d9aa8e0bf4f2198dbe697e6"} Jul 14 11:25:48.752808 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:48.752766 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" podStartSLOduration=1.710903387 podStartE2EDuration="3.752751431s" totalImagesPullingTime="2.041848044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:25:48.751677134 +0000 UTC m=+478.607511309" watchObservedRunningTime="2026-07-14 11:25:48.752751431 +0000 UTC m=+478.608585603" Jul 14 11:25:48.768348 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:48.768308 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" podStartSLOduration=2.32889668 podStartE2EDuration="3.768298041s" totalImagesPullingTime="1.439401361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:25:48.767243051 +0000 UTC m=+478.623077233" watchObservedRunningTime="2026-07-14 11:25:48.768298041 +0000 UTC m=+478.624132212" Jul 14 11:25:48.977197 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:48.977174 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:49.067289 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.067244 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w292\" (UniqueName: \"kubernetes.io/projected/0d5fac36-28e3-4eda-b863-77e472b4ae93-kube-api-access-2w292\") pod \"0d5fac36-28e3-4eda-b863-77e472b4ae93\" (UID: \"0d5fac36-28e3-4eda-b863-77e472b4ae93\") " Jul 14 11:25:49.069588 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.069561 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d5fac36-28e3-4eda-b863-77e472b4ae93-kube-api-access-2w292" (OuterVolumeSpecName: "kube-api-access-2w292") pod "0d5fac36-28e3-4eda-b863-77e472b4ae93" (UID: "0d5fac36-28e3-4eda-b863-77e472b4ae93"). InnerVolumeSpecName "kube-api-access-2w292". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:25:49.167761 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.167727 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w292\" (UniqueName: \"kubernetes.io/projected/0d5fac36-28e3-4eda-b863-77e472b4ae93-kube-api-access-2w292\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:25:49.742530 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.742493 2568 generic.go:356] "Generic (PLEG): container finished" podID="0d5fac36-28e3-4eda-b863-77e472b4ae93" containerID="2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47" exitCode=0 Jul 14 11:25:49.742946 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.742534 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" Jul 14 11:25:49.742946 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.742576 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" event={"ID":"0d5fac36-28e3-4eda-b863-77e472b4ae93","Type":"ContainerDied","Data":"2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47"} Jul 14 11:25:49.742946 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.742630 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n92fj" event={"ID":"0d5fac36-28e3-4eda-b863-77e472b4ae93","Type":"ContainerDied","Data":"f4bbb9c61af7030d0787fd0cd55dd8a3c4e23930519ca57bd1a603eec514a53c"} Jul 14 11:25:49.742946 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.742649 2568 scope.go:117] "RemoveContainer" containerID="2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47" Jul 14 11:25:49.751619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.751603 2568 scope.go:117] "RemoveContainer" containerID="2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47" Jul 14 11:25:49.751856 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:25:49.751839 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47\": container with ID starting with 2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47 not found: ID does not exist" containerID="2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47" Jul 14 11:25:49.751898 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.751864 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47"} err="failed to get container status \"2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47\": rpc error: code = NotFound desc = could not find container \"2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47\": container with ID starting with 2f58c0ed1f6bbf2a1a6a065f6756c81301bf1a947048f6d0babb92887acacc47 not found: ID does not exist" Jul 14 11:25:49.764606 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.764585 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n92fj"] Jul 14 11:25:49.768627 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:49.768606 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n92fj"] Jul 14 11:25:50.677527 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:50.677499 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d5fac36-28e3-4eda-b863-77e472b4ae93" path="/var/lib/kubelet/pods/0d5fac36-28e3-4eda-b863-77e472b4ae93/volumes" Jul 14 11:25:56.282439 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:56.282403 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:56.282439 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:56.282461 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:56.305227 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:56.305201 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:25:56.794782 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:25:56.794755 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2d2x4" Jul 14 11:26:18.552976 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.552941 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s"] Jul 14 11:26:18.553475 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.553279 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d5fac36-28e3-4eda-b863-77e472b4ae93" containerName="registry-server" Jul 14 11:26:18.553475 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.553291 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d5fac36-28e3-4eda-b863-77e472b4ae93" containerName="registry-server" Jul 14 11:26:18.553475 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.553358 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d5fac36-28e3-4eda-b863-77e472b4ae93" containerName="registry-server" Jul 14 11:26:18.559643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.559623 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.563011 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.562988 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-48pbg\"" Jul 14 11:26:18.570297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.570273 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s"] Jul 14 11:26:18.708591 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.708553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40bd68e3-2a2c-4993-afb5-3530d92e85c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.708751 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.708635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqz9x\" (UniqueName: \"kubernetes.io/projected/40bd68e3-2a2c-4993-afb5-3530d92e85c2-kube-api-access-cqz9x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.809401 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.809317 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40bd68e3-2a2c-4993-afb5-3530d92e85c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.809539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.809398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqz9x\" (UniqueName: \"kubernetes.io/projected/40bd68e3-2a2c-4993-afb5-3530d92e85c2-kube-api-access-cqz9x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.809714 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.809693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40bd68e3-2a2c-4993-afb5-3530d92e85c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.818708 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.818677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqz9x\" (UniqueName: \"kubernetes.io/projected/40bd68e3-2a2c-4993-afb5-3530d92e85c2-kube-api-access-cqz9x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:18.870087 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:18.870060 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:19.008380 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:26:19.008350 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bd68e3_2a2c_4993_afb5_3530d92e85c2.slice/crio-1be766ae7843cd03c20ed0bbebaf3ce99ca00fceada572af6aa94aeef8fd64ee WatchSource:0}: Error finding container 1be766ae7843cd03c20ed0bbebaf3ce99ca00fceada572af6aa94aeef8fd64ee: Status 404 returned error can't find the container with id 1be766ae7843cd03c20ed0bbebaf3ce99ca00fceada572af6aa94aeef8fd64ee Jul 14 11:26:19.009814 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:19.009791 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s"] Jul 14 11:26:19.873190 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:19.873100 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" event={"ID":"40bd68e3-2a2c-4993-afb5-3530d92e85c2","Type":"ContainerStarted","Data":"1be766ae7843cd03c20ed0bbebaf3ce99ca00fceada572af6aa94aeef8fd64ee"} Jul 14 11:26:20.537909 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.537872 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp"] Jul 14 11:26:20.541322 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.541305 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:20.544468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.544445 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rr8kp\"" Jul 14 11:26:20.554853 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.554833 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp"] Jul 14 11:26:20.625416 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.625384 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkxpd\" (UniqueName: \"kubernetes.io/projected/98b62b55-c816-4ff8-bb00-5b761b4443d7-kube-api-access-fkxpd\") pod \"limitador-operator-controller-manager-859d7b78bd-v27tp\" (UID: \"98b62b55-c816-4ff8-bb00-5b761b4443d7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:20.726009 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.725980 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkxpd\" (UniqueName: \"kubernetes.io/projected/98b62b55-c816-4ff8-bb00-5b761b4443d7-kube-api-access-fkxpd\") pod \"limitador-operator-controller-manager-859d7b78bd-v27tp\" (UID: \"98b62b55-c816-4ff8-bb00-5b761b4443d7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:20.750345 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.746799 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkxpd\" (UniqueName: \"kubernetes.io/projected/98b62b55-c816-4ff8-bb00-5b761b4443d7-kube-api-access-fkxpd\") pod \"limitador-operator-controller-manager-859d7b78bd-v27tp\" (UID: \"98b62b55-c816-4ff8-bb00-5b761b4443d7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:20.852454 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.852418 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:20.993707 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:20.993677 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp"] Jul 14 11:26:20.995754 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:26:20.995728 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b62b55_c816_4ff8_bb00_5b761b4443d7.slice/crio-1959aa8e9a88be292d61f8180816c10b9da0bc43eed41bb27131df1ed684da69 WatchSource:0}: Error finding container 1959aa8e9a88be292d61f8180816c10b9da0bc43eed41bb27131df1ed684da69: Status 404 returned error can't find the container with id 1959aa8e9a88be292d61f8180816c10b9da0bc43eed41bb27131df1ed684da69 Jul 14 11:26:21.884331 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:21.884291 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" event={"ID":"98b62b55-c816-4ff8-bb00-5b761b4443d7","Type":"ContainerStarted","Data":"1959aa8e9a88be292d61f8180816c10b9da0bc43eed41bb27131df1ed684da69"} Jul 14 11:26:22.890867 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.890831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" event={"ID":"98b62b55-c816-4ff8-bb00-5b761b4443d7","Type":"ContainerStarted","Data":"423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0"} Jul 14 11:26:22.891339 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.890900 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:22.913479 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.913426 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" podStartSLOduration=1.156646985 podStartE2EDuration="2.913410232s" totalImagesPullingTime="1.756763247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:22.911942467 +0000 UTC m=+512.767776641" watchObservedRunningTime="2026-07-14 11:26:22.913410232 +0000 UTC m=+512.769244403" Jul 14 11:26:22.953209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.953177 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qwt96"] Jul 14 11:26:22.955395 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.955379 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:22.958454 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.958431 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-29v8p\"" Jul 14 11:26:22.970014 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:22.969985 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qwt96"] Jul 14 11:26:23.048670 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:23.048640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wznvs\" (UniqueName: \"kubernetes.io/projected/83575570-bb14-4d4a-ace7-8665aed885e1-kube-api-access-wznvs\") pod \"authorino-operator-75cc8d5cb7-qwt96\" (UID: \"83575570-bb14-4d4a-ace7-8665aed885e1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:23.149274 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:23.149232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wznvs\" (UniqueName: \"kubernetes.io/projected/83575570-bb14-4d4a-ace7-8665aed885e1-kube-api-access-wznvs\") pod \"authorino-operator-75cc8d5cb7-qwt96\" (UID: \"83575570-bb14-4d4a-ace7-8665aed885e1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:23.158971 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:23.158937 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wznvs\" (UniqueName: \"kubernetes.io/projected/83575570-bb14-4d4a-ace7-8665aed885e1-kube-api-access-wznvs\") pod \"authorino-operator-75cc8d5cb7-qwt96\" (UID: \"83575570-bb14-4d4a-ace7-8665aed885e1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:23.264925 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:23.264888 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:23.416979 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:23.416931 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qwt96"] Jul 14 11:26:24.561118 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:26:24.561085 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83575570_bb14_4d4a_ace7_8665aed885e1.slice/crio-c8a3707784c87b8df9ef4f4cbd92c100b6e9dc9742c7a4f42f74a0f8c29b30fb WatchSource:0}: Error finding container c8a3707784c87b8df9ef4f4cbd92c100b6e9dc9742c7a4f42f74a0f8c29b30fb: Status 404 returned error can't find the container with id c8a3707784c87b8df9ef4f4cbd92c100b6e9dc9742c7a4f42f74a0f8c29b30fb Jul 14 11:26:24.903124 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:24.902980 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" event={"ID":"40bd68e3-2a2c-4993-afb5-3530d92e85c2","Type":"ContainerStarted","Data":"fa23443fa3ec4fd002337e2b44a349bf053ec889d4200fcfd82328d81dc86135"} Jul 14 11:26:24.903124 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:24.903061 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:24.904309 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:24.904286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" event={"ID":"83575570-bb14-4d4a-ace7-8665aed885e1","Type":"ContainerStarted","Data":"c8a3707784c87b8df9ef4f4cbd92c100b6e9dc9742c7a4f42f74a0f8c29b30fb"} Jul 14 11:26:24.931429 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:24.931382 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" podStartSLOduration=1.325749358 podStartE2EDuration="6.931362868s" totalImagesPullingTime="5.60561351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:24.928952177 +0000 UTC m=+514.784786350" watchObservedRunningTime="2026-07-14 11:26:24.931362868 +0000 UTC m=+514.787197042" Jul 14 11:26:26.919506 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:26.919468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" event={"ID":"83575570-bb14-4d4a-ace7-8665aed885e1","Type":"ContainerStarted","Data":"460a7d6d2b7413cbfc81bba127e516f7e07937712f568dd4afe8b33ab0be884d"} Jul 14 11:26:26.919882 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:26.919537 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:26.938803 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:26.938751 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" podStartSLOduration=2.782498981 podStartE2EDuration="4.938736421s" totalImagesPullingTime="2.15623744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:26.937132595 +0000 UTC m=+516.792966766" watchObservedRunningTime="2026-07-14 11:26:26.938736421 +0000 UTC m=+516.794570592" Jul 14 11:26:33.899297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:33.899247 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:35.913953 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:35.913923 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:37.827903 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.827866 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s"] Jul 14 11:26:37.828352 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.828033 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" containerName="manager" containerID="cri-o://fa23443fa3ec4fd002337e2b44a349bf053ec889d4200fcfd82328d81dc86135" gracePeriod=2 Jul 14 11:26:37.836516 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.836484 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s"] Jul 14 11:26:37.863203 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.863170 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp"] Jul 14 11:26:37.863441 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.863410 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" containerName="manager" containerID="cri-o://423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0" gracePeriod=2 Jul 14 11:26:37.867075 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:37.867046 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:37.883787 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.883761 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w"] Jul 14 11:26:37.884096 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.884084 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" containerName="manager" Jul 14 11:26:37.884096 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.884097 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" containerName="manager" Jul 14 11:26:37.884182 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.884168 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" containerName="manager" Jul 14 11:26:37.887014 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.886998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:37.892518 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.892497 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp"] Jul 14 11:26:37.902034 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.901840 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs"] Jul 14 11:26:37.902364 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.902345 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" containerName="manager" Jul 14 11:26:37.902364 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.902364 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" containerName="manager" Jul 14 11:26:37.902551 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.902450 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" containerName="manager" Jul 14 11:26:37.905207 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:37.905181 2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"98b62b55-c816-4ff8-bb00-5b761b4443d7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-v27tp\": pods \"limitador-operator-controller-manager-859d7b78bd-v27tp\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:37.905507 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.905487 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:37.908434 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:37.908408 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:37.914758 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:37.914729 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v27tp\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:37.917476 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.917451 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w"] Jul 14 11:26:37.925536 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.925515 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qwt96" Jul 14 11:26:37.947230 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.947202 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs"] Jul 14 11:26:37.972043 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:37.971997 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:37.975204 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:37.975121 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v27tp\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:37.975869 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.975797 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31820357-f74b-4a93-84de-ca01e2e78163-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xk52w\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:37.975942 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.975875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tk2m\" (UniqueName: \"kubernetes.io/projected/31820357-f74b-4a93-84de-ca01e2e78163-kube-api-access-6tk2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xk52w\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:37.976044 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.976021 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc47g\" (UniqueName: \"kubernetes.io/projected/598aa189-a6ed-4c6b-9dc2-289060fe9b74-kube-api-access-zc47g\") pod \"limitador-operator-controller-manager-859d7b78bd-br4xs\" (UID: \"598aa189-a6ed-4c6b-9dc2-289060fe9b74\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:37.979446 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:37.979421 2568 generic.go:356] "Generic (PLEG): container finished" podID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" containerID="fa23443fa3ec4fd002337e2b44a349bf053ec889d4200fcfd82328d81dc86135" exitCode=0 Jul 14 11:26:38.076889 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.076859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc47g\" (UniqueName: \"kubernetes.io/projected/598aa189-a6ed-4c6b-9dc2-289060fe9b74-kube-api-access-zc47g\") pod \"limitador-operator-controller-manager-859d7b78bd-br4xs\" (UID: \"598aa189-a6ed-4c6b-9dc2-289060fe9b74\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:38.077008 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.076921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31820357-f74b-4a93-84de-ca01e2e78163-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xk52w\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:38.077008 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.076962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tk2m\" (UniqueName: \"kubernetes.io/projected/31820357-f74b-4a93-84de-ca01e2e78163-kube-api-access-6tk2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xk52w\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:38.077391 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.077370 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31820357-f74b-4a93-84de-ca01e2e78163-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xk52w\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:38.099136 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.098981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tk2m\" (UniqueName: \"kubernetes.io/projected/31820357-f74b-4a93-84de-ca01e2e78163-kube-api-access-6tk2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xk52w\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:38.100838 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.100810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc47g\" (UniqueName: \"kubernetes.io/projected/598aa189-a6ed-4c6b-9dc2-289060fe9b74-kube-api-access-zc47g\") pod \"limitador-operator-controller-manager-859d7b78bd-br4xs\" (UID: \"598aa189-a6ed-4c6b-9dc2-289060fe9b74\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:38.127841 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.127820 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:38.129897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.129879 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:38.130915 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:38.130891 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:38.133592 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:38.133572 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v27tp\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:38.136346 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:38.136325 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d7h5s\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:38.141509 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:38.141487 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v27tp\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:38.177539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.177511 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40bd68e3-2a2c-4993-afb5-3530d92e85c2-extensions-socket-volume\") pod \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " Jul 14 11:26:38.177632 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.177572 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkxpd\" (UniqueName: \"kubernetes.io/projected/98b62b55-c816-4ff8-bb00-5b761b4443d7-kube-api-access-fkxpd\") pod \"98b62b55-c816-4ff8-bb00-5b761b4443d7\" (UID: \"98b62b55-c816-4ff8-bb00-5b761b4443d7\") " Jul 14 11:26:38.177687 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.177634 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqz9x\" (UniqueName: \"kubernetes.io/projected/40bd68e3-2a2c-4993-afb5-3530d92e85c2-kube-api-access-cqz9x\") pod \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\" (UID: \"40bd68e3-2a2c-4993-afb5-3530d92e85c2\") " Jul 14 11:26:38.178012 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.177990 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40bd68e3-2a2c-4993-afb5-3530d92e85c2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "40bd68e3-2a2c-4993-afb5-3530d92e85c2" (UID: "40bd68e3-2a2c-4993-afb5-3530d92e85c2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:38.179800 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.179777 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40bd68e3-2a2c-4993-afb5-3530d92e85c2-kube-api-access-cqz9x" (OuterVolumeSpecName: "kube-api-access-cqz9x") pod "40bd68e3-2a2c-4993-afb5-3530d92e85c2" (UID: "40bd68e3-2a2c-4993-afb5-3530d92e85c2"). InnerVolumeSpecName "kube-api-access-cqz9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:38.179884 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.179811 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b62b55-c816-4ff8-bb00-5b761b4443d7-kube-api-access-fkxpd" (OuterVolumeSpecName: "kube-api-access-fkxpd") pod "98b62b55-c816-4ff8-bb00-5b761b4443d7" (UID: "98b62b55-c816-4ff8-bb00-5b761b4443d7"). InnerVolumeSpecName "kube-api-access-fkxpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:38.263958 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.263929 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:38.272980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.272957 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:38.278887 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.278861 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqz9x\" (UniqueName: \"kubernetes.io/projected/40bd68e3-2a2c-4993-afb5-3530d92e85c2-kube-api-access-cqz9x\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:26:38.278961 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.278890 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40bd68e3-2a2c-4993-afb5-3530d92e85c2-extensions-socket-volume\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:26:38.278961 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.278901 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkxpd\" (UniqueName: \"kubernetes.io/projected/98b62b55-c816-4ff8-bb00-5b761b4443d7-kube-api-access-fkxpd\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:26:38.443919 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.443883 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w"] Jul 14 11:26:38.461491 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:26:38.461455 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31820357_f74b_4a93_84de_ca01e2e78163.slice/crio-83032a5cb87f94973592da4c480d04fc58faa74089b39451eb50097ab0a4f1ba WatchSource:0}: Error finding container 83032a5cb87f94973592da4c480d04fc58faa74089b39451eb50097ab0a4f1ba: Status 404 returned error can't find the container with id 83032a5cb87f94973592da4c480d04fc58faa74089b39451eb50097ab0a4f1ba Jul 14 11:26:38.485231 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.485205 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs"] Jul 14 11:26:38.499425 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:26:38.499401 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598aa189_a6ed_4c6b_9dc2_289060fe9b74.slice/crio-16a1e14d9359161a03f360d704944233ce3afd852b1311245a320c3e4872476c WatchSource:0}: Error finding container 16a1e14d9359161a03f360d704944233ce3afd852b1311245a320c3e4872476c: Status 404 returned error can't find the container with id 16a1e14d9359161a03f360d704944233ce3afd852b1311245a320c3e4872476c Jul 14 11:26:38.678292 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.678190 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40bd68e3-2a2c-4993-afb5-3530d92e85c2" path="/var/lib/kubelet/pods/40bd68e3-2a2c-4993-afb5-3530d92e85c2/volumes" Jul 14 11:26:38.678537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.678523 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b62b55-c816-4ff8-bb00-5b761b4443d7" path="/var/lib/kubelet/pods/98b62b55-c816-4ff8-bb00-5b761b4443d7/volumes" Jul 14 11:26:38.986523 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.986427 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" event={"ID":"31820357-f74b-4a93-84de-ca01e2e78163","Type":"ContainerStarted","Data":"2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630"} Jul 14 11:26:38.986523 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.986487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" event={"ID":"31820357-f74b-4a93-84de-ca01e2e78163","Type":"ContainerStarted","Data":"83032a5cb87f94973592da4c480d04fc58faa74089b39451eb50097ab0a4f1ba"} Jul 14 11:26:38.986523 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.986515 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:38.988091 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.988065 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" event={"ID":"598aa189-a6ed-4c6b-9dc2-289060fe9b74","Type":"ContainerStarted","Data":"4e92387c0415cc92ab377f34c19276222eabf7ea9e3c9b99de44ee66f62f626b"} Jul 14 11:26:38.988227 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.988111 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" event={"ID":"598aa189-a6ed-4c6b-9dc2-289060fe9b74","Type":"ContainerStarted","Data":"16a1e14d9359161a03f360d704944233ce3afd852b1311245a320c3e4872476c"} Jul 14 11:26:38.988227 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.988189 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:38.989412 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.989391 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d7h5s" Jul 14 11:26:38.989412 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.989401 2568 scope.go:117] "RemoveContainer" containerID="fa23443fa3ec4fd002337e2b44a349bf053ec889d4200fcfd82328d81dc86135" Jul 14 11:26:38.990775 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.990748 2568 generic.go:356] "Generic (PLEG): container finished" podID="98b62b55-c816-4ff8-bb00-5b761b4443d7" containerID="423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0" exitCode=0 Jul 14 11:26:38.990842 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.990783 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v27tp" Jul 14 11:26:38.998741 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:38.998718 2568 scope.go:117] "RemoveContainer" containerID="423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0" Jul 14 11:26:39.007013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:39.006995 2568 scope.go:117] "RemoveContainer" containerID="423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0" Jul 14 11:26:39.007319 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:39.007296 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0\": container with ID starting with 423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0 not found: ID does not exist" containerID="423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0" Jul 14 11:26:39.007418 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:39.007332 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0"} err="failed to get container status \"423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0\": rpc error: code = NotFound desc = could not find container \"423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0\": container with ID starting with 423a6bca36f68e570e96b5a8cee8841453fff1da1f5cf3d33e521d6aa0e385f0 not found: ID does not exist" Jul 14 11:26:39.019950 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:39.019902 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" podStartSLOduration=2.019889339 podStartE2EDuration="2.019889339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:39.016473988 +0000 UTC m=+528.872308172" watchObservedRunningTime="2026-07-14 11:26:39.019889339 +0000 UTC m=+528.875723510" Jul 14 11:26:39.065503 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:39.065458 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" podStartSLOduration=2.065444087 podStartE2EDuration="2.065444087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:39.064942153 +0000 UTC m=+528.920776327" watchObservedRunningTime="2026-07-14 11:26:39.065444087 +0000 UTC m=+528.921278259" Jul 14 11:26:40.098675 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.098642 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd"] Jul 14 11:26:40.103434 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.103417 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.143082 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.143054 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd"] Jul 14 11:26:40.193706 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.193679 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-kube-api-access-g8ftp\") pod \"kuadrant-operator-controller-manager-86878c769f-7xtzd\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.193706 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.193737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7xtzd\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.294186 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.294150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7xtzd\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.294470 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.294229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-kube-api-access-g8ftp\") pod \"kuadrant-operator-controller-manager-86878c769f-7xtzd\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.294632 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.294610 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7xtzd\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.316564 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.316535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-kube-api-access-g8ftp\") pod \"kuadrant-operator-controller-manager-86878c769f-7xtzd\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.413442 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.413364 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:40.563046 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:40.563020 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd"] Jul 14 11:26:40.565394 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:26:40.565367 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd75c9a_29ec_482a_b6d0_6d32a2ab2a94.slice/crio-f0613bed8277c6141ef809f9f19508c30d44f7e5d679cc4436185bfac18f2152 WatchSource:0}: Error finding container f0613bed8277c6141ef809f9f19508c30d44f7e5d679cc4436185bfac18f2152: Status 404 returned error can't find the container with id f0613bed8277c6141ef809f9f19508c30d44f7e5d679cc4436185bfac18f2152 Jul 14 11:26:41.007341 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:41.007298 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" event={"ID":"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94","Type":"ContainerStarted","Data":"a983240c7fd2363f59fcabfda787c1a8f366017fc40f648cff5321d4ac9f3cb2"} Jul 14 11:26:41.007537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:41.007400 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" event={"ID":"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94","Type":"ContainerStarted","Data":"f0613bed8277c6141ef809f9f19508c30d44f7e5d679cc4436185bfac18f2152"} Jul 14 11:26:41.007537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:41.007426 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:41.037051 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:41.036995 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" podStartSLOduration=1.036981018 podStartE2EDuration="1.036981018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:41.03685541 +0000 UTC m=+530.892689580" watchObservedRunningTime="2026-07-14 11:26:41.036981018 +0000 UTC m=+530.892815190" Jul 14 11:26:50.003319 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:50.003231 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:50.003684 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:50.003347 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-br4xs" Jul 14 11:26:52.014904 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.014876 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:26:52.074689 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.074660 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w"] Jul 14 11:26:52.074874 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.074850 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" podUID="31820357-f74b-4a93-84de-ca01e2e78163" containerName="manager" containerID="cri-o://2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630" gracePeriod=10 Jul 14 11:26:52.329430 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.329400 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:52.401212 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.401183 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tk2m\" (UniqueName: \"kubernetes.io/projected/31820357-f74b-4a93-84de-ca01e2e78163-kube-api-access-6tk2m\") pod \"31820357-f74b-4a93-84de-ca01e2e78163\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " Jul 14 11:26:52.401409 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.401308 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31820357-f74b-4a93-84de-ca01e2e78163-extensions-socket-volume\") pod \"31820357-f74b-4a93-84de-ca01e2e78163\" (UID: \"31820357-f74b-4a93-84de-ca01e2e78163\") " Jul 14 11:26:52.401661 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.401640 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31820357-f74b-4a93-84de-ca01e2e78163-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "31820357-f74b-4a93-84de-ca01e2e78163" (UID: "31820357-f74b-4a93-84de-ca01e2e78163"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:52.403442 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.403413 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31820357-f74b-4a93-84de-ca01e2e78163-kube-api-access-6tk2m" (OuterVolumeSpecName: "kube-api-access-6tk2m") pod "31820357-f74b-4a93-84de-ca01e2e78163" (UID: "31820357-f74b-4a93-84de-ca01e2e78163"). InnerVolumeSpecName "kube-api-access-6tk2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:52.502184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.502155 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tk2m\" (UniqueName: \"kubernetes.io/projected/31820357-f74b-4a93-84de-ca01e2e78163-kube-api-access-6tk2m\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:26:52.502184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:52.502181 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31820357-f74b-4a93-84de-ca01e2e78163-extensions-socket-volume\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:26:53.067836 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.067801 2568 generic.go:356] "Generic (PLEG): container finished" podID="31820357-f74b-4a93-84de-ca01e2e78163" containerID="2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630" exitCode=0 Jul 14 11:26:53.068235 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.067844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" event={"ID":"31820357-f74b-4a93-84de-ca01e2e78163","Type":"ContainerDied","Data":"2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630"} Jul 14 11:26:53.068235 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.067882 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" event={"ID":"31820357-f74b-4a93-84de-ca01e2e78163","Type":"ContainerDied","Data":"83032a5cb87f94973592da4c480d04fc58faa74089b39451eb50097ab0a4f1ba"} Jul 14 11:26:53.068235 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.067893 2568 scope.go:117] "RemoveContainer" containerID="2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630" Jul 14 11:26:53.068235 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.067879 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w" Jul 14 11:26:53.076963 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.076784 2568 scope.go:117] "RemoveContainer" containerID="2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630" Jul 14 11:26:53.077060 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:26:53.077040 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630\": container with ID starting with 2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630 not found: ID does not exist" containerID="2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630" Jul 14 11:26:53.077103 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.077070 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630"} err="failed to get container status \"2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630\": rpc error: code = NotFound desc = could not find container \"2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630\": container with ID starting with 2dacc987c5b3797ae7b5be5f059707120611fa0dfc6981037d31b92965480630 not found: ID does not exist" Jul 14 11:26:53.121340 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.121315 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w"] Jul 14 11:26:53.127715 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:53.127693 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xk52w"] Jul 14 11:26:54.677153 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:26:54.677119 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31820357-f74b-4a93-84de-ca01e2e78163" path="/var/lib/kubelet/pods/31820357-f74b-4a93-84de-ca01e2e78163/volumes" Jul 14 11:27:10.907229 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.907189 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:10.907790 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.907719 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31820357-f74b-4a93-84de-ca01e2e78163" containerName="manager" Jul 14 11:27:10.907790 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.907739 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="31820357-f74b-4a93-84de-ca01e2e78163" containerName="manager" Jul 14 11:27:10.907900 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.907839 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="31820357-f74b-4a93-84de-ca01e2e78163" containerName="manager" Jul 14 11:27:10.912320 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.912297 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:10.915561 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.915538 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-82t9v\"" Jul 14 11:27:10.915688 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.915538 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 11:27:10.920537 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.920519 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:10.998589 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:10.998560 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:11.057547 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.057512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj5nw\" (UniqueName: \"kubernetes.io/projected/8acb5495-3a14-4d04-8866-94d46575e0f6-kube-api-access-gj5nw\") pod \"limitador-limitador-64c94fd64f-qp87h\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.057726 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.057561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8acb5495-3a14-4d04-8866-94d46575e0f6-config-file\") pod \"limitador-limitador-64c94fd64f-qp87h\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.158437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.158353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj5nw\" (UniqueName: \"kubernetes.io/projected/8acb5495-3a14-4d04-8866-94d46575e0f6-kube-api-access-gj5nw\") pod \"limitador-limitador-64c94fd64f-qp87h\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.158437 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.158399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8acb5495-3a14-4d04-8866-94d46575e0f6-config-file\") pod \"limitador-limitador-64c94fd64f-qp87h\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.159008 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.158991 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8acb5495-3a14-4d04-8866-94d46575e0f6-config-file\") pod \"limitador-limitador-64c94fd64f-qp87h\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.167646 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.167616 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj5nw\" (UniqueName: \"kubernetes.io/projected/8acb5495-3a14-4d04-8866-94d46575e0f6-kube-api-access-gj5nw\") pod \"limitador-limitador-64c94fd64f-qp87h\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.223174 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.223136 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:11.363059 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.363037 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:11.365381 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:27:11.365356 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acb5495_3a14_4d04_8866_94d46575e0f6.slice/crio-012236b5410aecaaf9db94f79dfcf24acdfbfe56ac2f72d3198b2e1ba67645f6 WatchSource:0}: Error finding container 012236b5410aecaaf9db94f79dfcf24acdfbfe56ac2f72d3198b2e1ba67645f6: Status 404 returned error can't find the container with id 012236b5410aecaaf9db94f79dfcf24acdfbfe56ac2f72d3198b2e1ba67645f6 Jul 14 11:27:11.831645 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.831602 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-sgtzv"] Jul 14 11:27:11.836069 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.836042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sgtzv"] Jul 14 11:27:11.836199 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.836154 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:11.839222 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.839198 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-s64ln\"" Jul 14 11:27:11.965084 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:11.965055 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwn2n\" (UniqueName: \"kubernetes.io/projected/149e745b-d3c7-42f1-ba1e-b5c4c0035b73-kube-api-access-vwn2n\") pod \"authorino-75d556bd77-sgtzv\" (UID: \"149e745b-d3c7-42f1-ba1e-b5c4c0035b73\") " pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:12.066393 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:12.066359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwn2n\" (UniqueName: \"kubernetes.io/projected/149e745b-d3c7-42f1-ba1e-b5c4c0035b73-kube-api-access-vwn2n\") pod \"authorino-75d556bd77-sgtzv\" (UID: \"149e745b-d3c7-42f1-ba1e-b5c4c0035b73\") " pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:12.080225 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:12.079734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwn2n\" (UniqueName: \"kubernetes.io/projected/149e745b-d3c7-42f1-ba1e-b5c4c0035b73-kube-api-access-vwn2n\") pod \"authorino-75d556bd77-sgtzv\" (UID: \"149e745b-d3c7-42f1-ba1e-b5c4c0035b73\") " pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:12.146408 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:12.146320 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:12.171391 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:12.168867 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" event={"ID":"8acb5495-3a14-4d04-8866-94d46575e0f6","Type":"ContainerStarted","Data":"012236b5410aecaaf9db94f79dfcf24acdfbfe56ac2f72d3198b2e1ba67645f6"} Jul 14 11:27:12.355438 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:12.355402 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sgtzv"] Jul 14 11:27:13.176198 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:13.176150 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sgtzv" event={"ID":"149e745b-d3c7-42f1-ba1e-b5c4c0035b73","Type":"ContainerStarted","Data":"e500b941f6dc7aa66354ede8373acdaf6e0b8bf6315b7b0c9d52844d7ef19acb"} Jul 14 11:27:15.191943 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:15.191899 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" event={"ID":"8acb5495-3a14-4d04-8866-94d46575e0f6","Type":"ContainerStarted","Data":"b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c"} Jul 14 11:27:15.192424 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:15.192082 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:15.228979 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:15.228917 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" podStartSLOduration=2.493455548 podStartE2EDuration="5.228901833s" totalImagesPullingTime="2.735446285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:15.227905997 +0000 UTC m=+565.083740168" watchObservedRunningTime="2026-07-14 11:27:15.228901833 +0000 UTC m=+565.084736003" Jul 14 11:27:17.204324 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:17.204284 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sgtzv" event={"ID":"149e745b-d3c7-42f1-ba1e-b5c4c0035b73","Type":"ContainerStarted","Data":"e85d384ae8ae8e0edd09dadb5a29733894eef0128e750c3996f498e0b9e899e3"} Jul 14 11:27:17.220802 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:17.220756 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-sgtzv" podStartSLOduration=2.112864904 podStartE2EDuration="6.220740948s" totalImagesPullingTime="4.107876044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:17.219720121 +0000 UTC m=+567.075554297" watchObservedRunningTime="2026-07-14 11:27:17.220740948 +0000 UTC m=+567.076575119" Jul 14 11:27:26.065218 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.065180 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:26.065696 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.065386 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" podUID="8acb5495-3a14-4d04-8866-94d46575e0f6" containerName="limitador" containerID="cri-o://b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c" gracePeriod=30 Jul 14 11:27:26.066137 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.066116 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:26.624377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.624313 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:26.794343 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.794312 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj5nw\" (UniqueName: \"kubernetes.io/projected/8acb5495-3a14-4d04-8866-94d46575e0f6-kube-api-access-gj5nw\") pod \"8acb5495-3a14-4d04-8866-94d46575e0f6\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " Jul 14 11:27:26.794593 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.794402 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8acb5495-3a14-4d04-8866-94d46575e0f6-config-file\") pod \"8acb5495-3a14-4d04-8866-94d46575e0f6\" (UID: \"8acb5495-3a14-4d04-8866-94d46575e0f6\") " Jul 14 11:27:26.794840 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.794804 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8acb5495-3a14-4d04-8866-94d46575e0f6-config-file" (OuterVolumeSpecName: "config-file") pod "8acb5495-3a14-4d04-8866-94d46575e0f6" (UID: "8acb5495-3a14-4d04-8866-94d46575e0f6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:27:26.796710 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.796687 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8acb5495-3a14-4d04-8866-94d46575e0f6-kube-api-access-gj5nw" (OuterVolumeSpecName: "kube-api-access-gj5nw") pod "8acb5495-3a14-4d04-8866-94d46575e0f6" (UID: "8acb5495-3a14-4d04-8866-94d46575e0f6"). InnerVolumeSpecName "kube-api-access-gj5nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:27:26.896029 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.895947 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8acb5495-3a14-4d04-8866-94d46575e0f6-config-file\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:27:26.896029 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:26.895975 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj5nw\" (UniqueName: \"kubernetes.io/projected/8acb5495-3a14-4d04-8866-94d46575e0f6-kube-api-access-gj5nw\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:27:27.250596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.250509 2568 generic.go:356] "Generic (PLEG): container finished" podID="8acb5495-3a14-4d04-8866-94d46575e0f6" containerID="b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c" exitCode=0 Jul 14 11:27:27.250596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.250548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" event={"ID":"8acb5495-3a14-4d04-8866-94d46575e0f6","Type":"ContainerDied","Data":"b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c"} Jul 14 11:27:27.250596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.250554 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" Jul 14 11:27:27.250596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.250592 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qp87h" event={"ID":"8acb5495-3a14-4d04-8866-94d46575e0f6","Type":"ContainerDied","Data":"012236b5410aecaaf9db94f79dfcf24acdfbfe56ac2f72d3198b2e1ba67645f6"} Jul 14 11:27:27.251117 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.250614 2568 scope.go:117] "RemoveContainer" containerID="b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c" Jul 14 11:27:27.259403 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.259379 2568 scope.go:117] "RemoveContainer" containerID="b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c" Jul 14 11:27:27.259663 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:27:27.259642 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c\": container with ID starting with b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c not found: ID does not exist" containerID="b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c" Jul 14 11:27:27.259707 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.259672 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c"} err="failed to get container status \"b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c\": rpc error: code = NotFound desc = could not find container \"b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c\": container with ID starting with b14a9659870126725ee3d2fcf13cff8bf93d4c9839f199a52b7de8a1790fac1c not found: ID does not exist" Jul 14 11:27:27.291277 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.290580 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:27.295904 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.293504 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qp87h"] Jul 14 11:27:27.833581 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.833549 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5"] Jul 14 11:27:27.833905 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.833893 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8acb5495-3a14-4d04-8866-94d46575e0f6" containerName="limitador" Jul 14 11:27:27.833947 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.833906 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8acb5495-3a14-4d04-8866-94d46575e0f6" containerName="limitador" Jul 14 11:27:27.833980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.833971 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8acb5495-3a14-4d04-8866-94d46575e0f6" containerName="limitador" Jul 14 11:27:27.838226 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.838209 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:27.845606 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.845587 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 14 11:27:27.845730 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.845608 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 11:27:27.847274 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.847208 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 11:27:27.847346 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.847294 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gq9l5\"" Jul 14 11:27:27.908405 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:27.908370 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5"] Jul 14 11:27:28.003993 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.003957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ebf43b49-05a0-4536-adbe-87018330f062-data\") pod \"postgres-69fb4594cf-nvzt5\" (UID: \"ebf43b49-05a0-4536-adbe-87018330f062\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.004164 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.004007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9czgb\" (UniqueName: \"kubernetes.io/projected/ebf43b49-05a0-4536-adbe-87018330f062-kube-api-access-9czgb\") pod \"postgres-69fb4594cf-nvzt5\" (UID: \"ebf43b49-05a0-4536-adbe-87018330f062\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.104575 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.104484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ebf43b49-05a0-4536-adbe-87018330f062-data\") pod \"postgres-69fb4594cf-nvzt5\" (UID: \"ebf43b49-05a0-4536-adbe-87018330f062\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.104575 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.104525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9czgb\" (UniqueName: \"kubernetes.io/projected/ebf43b49-05a0-4536-adbe-87018330f062-kube-api-access-9czgb\") pod \"postgres-69fb4594cf-nvzt5\" (UID: \"ebf43b49-05a0-4536-adbe-87018330f062\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.104898 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.104876 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ebf43b49-05a0-4536-adbe-87018330f062-data\") pod \"postgres-69fb4594cf-nvzt5\" (UID: \"ebf43b49-05a0-4536-adbe-87018330f062\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.115192 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.115164 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9czgb\" (UniqueName: \"kubernetes.io/projected/ebf43b49-05a0-4536-adbe-87018330f062-kube-api-access-9czgb\") pod \"postgres-69fb4594cf-nvzt5\" (UID: \"ebf43b49-05a0-4536-adbe-87018330f062\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.148740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.148703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:28.284938 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.284815 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5"] Jul 14 11:27:28.288284 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:27:28.288238 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf43b49_05a0_4536_adbe_87018330f062.slice/crio-eec85eb386ae00c774bdd05f10b49331b380f5168c726d70d1d653dfe66bd88c WatchSource:0}: Error finding container eec85eb386ae00c774bdd05f10b49331b380f5168c726d70d1d653dfe66bd88c: Status 404 returned error can't find the container with id eec85eb386ae00c774bdd05f10b49331b380f5168c726d70d1d653dfe66bd88c Jul 14 11:27:28.678609 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:28.678577 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8acb5495-3a14-4d04-8866-94d46575e0f6" path="/var/lib/kubelet/pods/8acb5495-3a14-4d04-8866-94d46575e0f6/volumes" Jul 14 11:27:29.265738 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:29.265667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" event={"ID":"ebf43b49-05a0-4536-adbe-87018330f062","Type":"ContainerStarted","Data":"eec85eb386ae00c774bdd05f10b49331b380f5168c726d70d1d653dfe66bd88c"} Jul 14 11:27:33.287272 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:33.287165 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" event={"ID":"ebf43b49-05a0-4536-adbe-87018330f062","Type":"ContainerStarted","Data":"e80ba3349197c45875c69d1d338144972b5601ad294c6284e870a902f12444d6"} Jul 14 11:27:33.287622 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:33.287278 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:33.310813 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:33.310751 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" podStartSLOduration=1.7083697949999999 podStartE2EDuration="6.310732502s" totalImagesPullingTime="4.602362707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:33.310423318 +0000 UTC m=+583.166257494" watchObservedRunningTime="2026-07-14 11:27:33.310732502 +0000 UTC m=+583.166566674" Jul 14 11:27:39.322522 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:39.322491 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nvzt5" Jul 14 11:27:42.181566 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.181529 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sgtzv"] Jul 14 11:27:42.182004 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.181759 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-sgtzv" podUID="149e745b-d3c7-42f1-ba1e-b5c4c0035b73" containerName="authorino" containerID="cri-o://e85d384ae8ae8e0edd09dadb5a29733894eef0128e750c3996f498e0b9e899e3" gracePeriod=30 Jul 14 11:27:42.334219 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.334185 2568 generic.go:356] "Generic (PLEG): container finished" podID="149e745b-d3c7-42f1-ba1e-b5c4c0035b73" containerID="e85d384ae8ae8e0edd09dadb5a29733894eef0128e750c3996f498e0b9e899e3" exitCode=0 Jul 14 11:27:42.334418 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.334277 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sgtzv" event={"ID":"149e745b-d3c7-42f1-ba1e-b5c4c0035b73","Type":"ContainerDied","Data":"e85d384ae8ae8e0edd09dadb5a29733894eef0128e750c3996f498e0b9e899e3"} Jul 14 11:27:42.443036 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.442961 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:42.520847 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.520806 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwn2n\" (UniqueName: \"kubernetes.io/projected/149e745b-d3c7-42f1-ba1e-b5c4c0035b73-kube-api-access-vwn2n\") pod \"149e745b-d3c7-42f1-ba1e-b5c4c0035b73\" (UID: \"149e745b-d3c7-42f1-ba1e-b5c4c0035b73\") " Jul 14 11:27:42.522989 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.522960 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149e745b-d3c7-42f1-ba1e-b5c4c0035b73-kube-api-access-vwn2n" (OuterVolumeSpecName: "kube-api-access-vwn2n") pod "149e745b-d3c7-42f1-ba1e-b5c4c0035b73" (UID: "149e745b-d3c7-42f1-ba1e-b5c4c0035b73"). InnerVolumeSpecName "kube-api-access-vwn2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:27:42.621539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:42.621499 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwn2n\" (UniqueName: \"kubernetes.io/projected/149e745b-d3c7-42f1-ba1e-b5c4c0035b73-kube-api-access-vwn2n\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:27:43.339819 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:43.339788 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sgtzv" Jul 14 11:27:43.340322 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:43.339819 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sgtzv" event={"ID":"149e745b-d3c7-42f1-ba1e-b5c4c0035b73","Type":"ContainerDied","Data":"e500b941f6dc7aa66354ede8373acdaf6e0b8bf6315b7b0c9d52844d7ef19acb"} Jul 14 11:27:43.340322 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:43.339876 2568 scope.go:117] "RemoveContainer" containerID="e85d384ae8ae8e0edd09dadb5a29733894eef0128e750c3996f498e0b9e899e3" Jul 14 11:27:43.357552 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:43.357519 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sgtzv"] Jul 14 11:27:43.361738 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:43.361715 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sgtzv"] Jul 14 11:27:44.678032 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:44.677991 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="149e745b-d3c7-42f1-ba1e-b5c4c0035b73" path="/var/lib/kubelet/pods/149e745b-d3c7-42f1-ba1e-b5c4c0035b73/volumes" Jul 14 11:27:50.589568 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:50.589536 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:27:50.591061 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:50.591041 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:27:50.596840 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:50.596821 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:27:50.599006 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:27:50.598979 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:28:19.973099 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.973023 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-rpmgm"] Jul 14 11:28:19.973494 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.973428 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149e745b-d3c7-42f1-ba1e-b5c4c0035b73" containerName="authorino" Jul 14 11:28:19.973494 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.973441 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="149e745b-d3c7-42f1-ba1e-b5c4c0035b73" containerName="authorino" Jul 14 11:28:19.973571 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.973518 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="149e745b-d3c7-42f1-ba1e-b5c4c0035b73" containerName="authorino" Jul 14 11:28:19.976466 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.976446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:19.979451 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.979431 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 11:28:19.987966 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:19.987943 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-rpmgm"] Jul 14 11:28:20.039528 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.039498 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-qprhx"] Jul 14 11:28:20.042790 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.042771 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.045681 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.045655 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jpzxw\"" Jul 14 11:28:20.053440 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.053417 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-qprhx"] Jul 14 11:28:20.131116 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.131083 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-rpmgm\" (UID: \"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.131297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.131132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb482966-a541-4ffb-a8d9-3745dad2fde3-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-qprhx\" (UID: \"bb482966-a541-4ffb-a8d9-3745dad2fde3\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.131297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.131181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c-kube-api-access-lfhcr\") pod \"payload-processing-f9b8d9b9b-rpmgm\" (UID: \"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.131297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.131278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2p8\" (UniqueName: \"kubernetes.io/projected/bb482966-a541-4ffb-a8d9-3745dad2fde3-kube-api-access-wp2p8\") pod \"payload-pre-processing-74584bb9f8-qprhx\" (UID: \"bb482966-a541-4ffb-a8d9-3745dad2fde3\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.232662 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.232583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-rpmgm\" (UID: \"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.232662 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.232629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb482966-a541-4ffb-a8d9-3745dad2fde3-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-qprhx\" (UID: \"bb482966-a541-4ffb-a8d9-3745dad2fde3\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.232662 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.232652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c-kube-api-access-lfhcr\") pod \"payload-processing-f9b8d9b9b-rpmgm\" (UID: \"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.232884 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.232687 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2p8\" (UniqueName: \"kubernetes.io/projected/bb482966-a541-4ffb-a8d9-3745dad2fde3-kube-api-access-wp2p8\") pod \"payload-pre-processing-74584bb9f8-qprhx\" (UID: \"bb482966-a541-4ffb-a8d9-3745dad2fde3\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.233384 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.233364 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-rpmgm\" (UID: \"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.233426 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.233404 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb482966-a541-4ffb-a8d9-3745dad2fde3-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-qprhx\" (UID: \"bb482966-a541-4ffb-a8d9-3745dad2fde3\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.245464 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.245437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2p8\" (UniqueName: \"kubernetes.io/projected/bb482966-a541-4ffb-a8d9-3745dad2fde3-kube-api-access-wp2p8\") pod \"payload-pre-processing-74584bb9f8-qprhx\" (UID: \"bb482966-a541-4ffb-a8d9-3745dad2fde3\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.245629 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.245611 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhcr\" (UniqueName: \"kubernetes.io/projected/ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c-kube-api-access-lfhcr\") pod \"payload-processing-f9b8d9b9b-rpmgm\" (UID: \"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.286611 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.286577 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:20.354962 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.354928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:20.432703 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.432671 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-rpmgm"] Jul 14 11:28:20.435667 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:28:20.435640 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac13aa1f_5d9a_45c5_95e0_4ee3d06d084c.slice/crio-4b867e6f35e4629df90bbe1a4417100ad7d6580d50ce6e3f3feee60b300cbeb3 WatchSource:0}: Error finding container 4b867e6f35e4629df90bbe1a4417100ad7d6580d50ce6e3f3feee60b300cbeb3: Status 404 returned error can't find the container with id 4b867e6f35e4629df90bbe1a4417100ad7d6580d50ce6e3f3feee60b300cbeb3 Jul 14 11:28:20.500953 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.500916 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-qprhx"] Jul 14 11:28:20.503221 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:28:20.503191 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb482966_a541_4ffb_a8d9_3745dad2fde3.slice/crio-805ba9128db94e88c627ba87e5bddb633f5480b07de3a6c908a396d3fe53e745 WatchSource:0}: Error finding container 805ba9128db94e88c627ba87e5bddb633f5480b07de3a6c908a396d3fe53e745: Status 404 returned error can't find the container with id 805ba9128db94e88c627ba87e5bddb633f5480b07de3a6c908a396d3fe53e745 Jul 14 11:28:20.510509 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.510476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" event={"ID":"bb482966-a541-4ffb-a8d9-3745dad2fde3","Type":"ContainerStarted","Data":"805ba9128db94e88c627ba87e5bddb633f5480b07de3a6c908a396d3fe53e745"} Jul 14 11:28:20.511662 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:20.511641 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" event={"ID":"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c","Type":"ContainerStarted","Data":"4b867e6f35e4629df90bbe1a4417100ad7d6580d50ce6e3f3feee60b300cbeb3"} Jul 14 11:28:22.523269 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:22.523213 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" event={"ID":"bb482966-a541-4ffb-a8d9-3745dad2fde3","Type":"ContainerStarted","Data":"51f2b161ca5a207d84c27b6d18fbcb0afd64f6335578604e02945a6d7f134d36"} Jul 14 11:28:22.523788 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:22.523417 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:22.524828 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:22.524804 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" event={"ID":"ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c","Type":"ContainerStarted","Data":"6cc4257a9e64093805bada4f5d7722ab8185610a6d80810b471c79473ed65512"} Jul 14 11:28:22.524947 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:22.524855 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:22.540276 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:22.540214 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" podStartSLOduration=1.299296334 podStartE2EDuration="2.540201634s" totalImagesPullingTime="1.2409053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:28:22.539831324 +0000 UTC m=+632.395665497" watchObservedRunningTime="2026-07-14 11:28:22.540201634 +0000 UTC m=+632.396035806" Jul 14 11:28:22.555734 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:22.555688 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" podStartSLOduration=2.244427715 podStartE2EDuration="3.555675824s" totalImagesPullingTime="1.311248109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:28:22.555108892 +0000 UTC m=+632.410943063" watchObservedRunningTime="2026-07-14 11:28:22.555675824 +0000 UTC m=+632.411509996" Jul 14 11:28:33.532188 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:33.532157 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-rpmgm" Jul 14 11:28:33.534572 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:33.532224 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qprhx" Jul 14 11:28:55.275127 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.275083 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs"] Jul 14 11:28:55.278900 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.278882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.282393 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.282374 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 11:28:55.282498 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.282375 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 11:28:55.283470 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.283455 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vkrbv\"" Jul 14 11:28:55.283470 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.283469 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 14 11:28:55.289013 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.288992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs"] Jul 14 11:28:55.329995 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.329967 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.330155 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.330009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fj9f\" (UniqueName: \"kubernetes.io/projected/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-kube-api-access-4fj9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.330155 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.330033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.330155 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.330094 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.330278 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.330166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.330278 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.330200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431574 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431448 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431574 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431695 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fj9f\" (UniqueName: \"kubernetes.io/projected/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-kube-api-access-4fj9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431695 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431695 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431911 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431888 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.431982 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.431942 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.432039 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.432002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.433950 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.433920 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.434134 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.434118 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.441203 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.441178 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fj9f\" (UniqueName: \"kubernetes.io/projected/0a1973b4-28dc-4b26-bf32-e00f8b5f1439-kube-api-access-4fj9f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs\" (UID: \"0a1973b4-28dc-4b26-bf32-e00f8b5f1439\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.589407 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.589359 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:28:55.730421 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:55.730396 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs"] Jul 14 11:28:55.732700 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:28:55.732667 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1973b4_28dc_4b26_bf32_e00f8b5f1439.slice/crio-594003c0649d198b99434b883ee9c82fcf4bd6e73e52c7ee1a9918dedd62d96d WatchSource:0}: Error finding container 594003c0649d198b99434b883ee9c82fcf4bd6e73e52c7ee1a9918dedd62d96d: Status 404 returned error can't find the container with id 594003c0649d198b99434b883ee9c82fcf4bd6e73e52c7ee1a9918dedd62d96d Jul 14 11:28:56.698478 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:56.698439 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" event={"ID":"0a1973b4-28dc-4b26-bf32-e00f8b5f1439","Type":"ContainerStarted","Data":"594003c0649d198b99434b883ee9c82fcf4bd6e73e52c7ee1a9918dedd62d96d"} Jul 14 11:28:57.705009 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:57.704980 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" event={"ID":"0a1973b4-28dc-4b26-bf32-e00f8b5f1439","Type":"ContainerStarted","Data":"0317ed01b2964e0bef1d1bf4229c4518d2617dcad64c8575c5a5b1b257d6cd45"} Jul 14 11:28:57.727947 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:28:57.727898 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" podStartSLOduration=1.693156428 podStartE2EDuration="2.727882121s" totalImagesPullingTime="1.034725693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:28:57.727580624 +0000 UTC m=+667.583414795" watchObservedRunningTime="2026-07-14 11:28:57.727882121 +0000 UTC m=+667.583716292" Jul 14 11:29:05.590295 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:05.590227 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:29:05.590707 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:05.590319 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:29:05.603272 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:05.603218 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:29:05.754849 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:05.754823 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs" Jul 14 11:29:37.861683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.861653 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4"] Jul 14 11:29:37.867594 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.867573 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:37.870975 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.870955 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 11:29:37.878337 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.878310 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4"] Jul 14 11:29:37.905470 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.905441 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbrhm\" (UniqueName: \"kubernetes.io/projected/ebb222cb-761a-4ff2-ab0a-9b7c98694780-kube-api-access-rbrhm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:37.905639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.905493 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:37.905639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.905510 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebb222cb-761a-4ff2-ab0a-9b7c98694780-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:37.905639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.905535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:37.905639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.905560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:37.905639 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:37.905585 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006022 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.005989 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006022 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebb222cb-761a-4ff2-ab0a-9b7c98694780-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006074 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006377 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006184 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrhm\" (UniqueName: \"kubernetes.io/projected/ebb222cb-761a-4ff2-ab0a-9b7c98694780-kube-api-access-rbrhm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006579 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006452 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006579 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.006647 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.006579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.009168 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.009139 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebb222cb-761a-4ff2-ab0a-9b7c98694780-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.009402 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.009382 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebb222cb-761a-4ff2-ab0a-9b7c98694780-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.017057 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.017033 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbrhm\" (UniqueName: \"kubernetes.io/projected/ebb222cb-761a-4ff2-ab0a-9b7c98694780-kube-api-access-rbrhm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2fz4\" (UID: \"ebb222cb-761a-4ff2-ab0a-9b7c98694780\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.179210 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.179114 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:38.321326 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.321299 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4"] Jul 14 11:29:38.322659 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:29:38.322630 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb222cb_761a_4ff2_ab0a_9b7c98694780.slice/crio-07a01ef200e4798fb3813902e2d36b91a22800e6880f3924408658581228b192 WatchSource:0}: Error finding container 07a01ef200e4798fb3813902e2d36b91a22800e6880f3924408658581228b192: Status 404 returned error can't find the container with id 07a01ef200e4798fb3813902e2d36b91a22800e6880f3924408658581228b192 Jul 14 11:29:38.324425 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.324405 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:29:38.899619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.899583 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" event={"ID":"ebb222cb-761a-4ff2-ab0a-9b7c98694780","Type":"ContainerStarted","Data":"901cbf6a73e79493fb8f12c5d3728208ccb0b27e072a51e5ff025002bd1f78b4"} Jul 14 11:29:38.900009 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.899644 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" event={"ID":"ebb222cb-761a-4ff2-ab0a-9b7c98694780","Type":"ContainerStarted","Data":"07a01ef200e4798fb3813902e2d36b91a22800e6880f3924408658581228b192"} Jul 14 11:29:38.924452 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:38.924405 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" podStartSLOduration=1.753625509 podStartE2EDuration="1.924389898s" totalImagesPullingTime="170.764389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:29:38.920668167 +0000 UTC m=+708.776502338" watchObservedRunningTime="2026-07-14 11:29:38.924389898 +0000 UTC m=+708.780224070" Jul 14 11:29:48.179861 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:48.179775 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:48.179861 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:48.179861 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:48.192344 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:48.192321 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:29:48.961370 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:29:48.961344 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2fz4" Jul 14 11:30:00.143812 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:00.143778 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m"] Jul 14 11:30:00.147457 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:00.147438 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" Jul 14 11:30:00.150364 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:00.150344 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\"" Jul 14 11:30:00.155598 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:00.155575 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m"] Jul 14 11:30:00.158390 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:00.158372 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" Jul 14 11:30:00.293208 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:00.293153 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m"] Jul 14 11:30:00.295743 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:30:00.295714 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb408680_76f3_432e_b2e7_a2c1a47c40ca.slice/crio-cd4dd3adc4ce882b1d083b7f45f4a795055cec12d19b83e98ea6023ba61029d4 WatchSource:0}: Error finding container cd4dd3adc4ce882b1d083b7f45f4a795055cec12d19b83e98ea6023ba61029d4: Status 404 returned error can't find the container with id cd4dd3adc4ce882b1d083b7f45f4a795055cec12d19b83e98ea6023ba61029d4 Jul 14 11:30:01.008851 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:01.008817 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" event={"ID":"cb408680-76f3-432e-b2e7-a2c1a47c40ca","Type":"ContainerStarted","Data":"cd4dd3adc4ce882b1d083b7f45f4a795055cec12d19b83e98ea6023ba61029d4"} Jul 14 11:30:03.021388 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:03.021301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" event={"ID":"cb408680-76f3-432e-b2e7-a2c1a47c40ca","Type":"ContainerStarted","Data":"34947977b6645ada9caafe1e7d77b900d2e7129df128e2c3a8cd1c7aa3085a3d"} Jul 14 11:30:03.038876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:03.038825 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" podStartSLOduration=0.979295934 podStartE2EDuration="3.038810619s" totalImagesPullingTime="2.059514685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:30:03.037132155 +0000 UTC m=+732.892966323" watchObservedRunningTime="2026-07-14 11:30:03.038810619 +0000 UTC m=+732.894644792" Jul 14 11:30:04.028037 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:04.028003 2568 generic.go:356] "Generic (PLEG): container finished" podID="cb408680-76f3-432e-b2e7-a2c1a47c40ca" containerID="34947977b6645ada9caafe1e7d77b900d2e7129df128e2c3a8cd1c7aa3085a3d" exitCode=0 Jul 14 11:30:04.028417 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:04.028086 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" event={"ID":"cb408680-76f3-432e-b2e7-a2c1a47c40ca","Type":"ContainerDied","Data":"34947977b6645ada9caafe1e7d77b900d2e7129df128e2c3a8cd1c7aa3085a3d"} Jul 14 11:30:05.168592 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:05.168566 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" Jul 14 11:30:06.039012 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:06.038977 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" Jul 14 11:30:06.039191 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:06.039014 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733810-lbh2m" event={"ID":"cb408680-76f3-432e-b2e7-a2c1a47c40ca","Type":"ContainerDied","Data":"cd4dd3adc4ce882b1d083b7f45f4a795055cec12d19b83e98ea6023ba61029d4"} Jul 14 11:30:06.039191 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:30:06.039069 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd4dd3adc4ce882b1d083b7f45f4a795055cec12d19b83e98ea6023ba61029d4" Jul 14 11:32:50.622729 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:32:50.622702 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:32:50.623230 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:32:50.623220 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:32:50.630595 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:32:50.630564 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:32:50.631006 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:32:50.630989 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:36:26.302836 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:26.302798 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd"] Jul 14 11:36:26.303359 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:26.303046 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" podUID="3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" containerName="manager" containerID="cri-o://a983240c7fd2363f59fcabfda787c1a8f366017fc40f648cff5321d4ac9f3cb2" gracePeriod=10 Jul 14 11:36:28.960839 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:28.960807 2568 generic.go:356] "Generic (PLEG): container finished" podID="3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" containerID="a983240c7fd2363f59fcabfda787c1a8f366017fc40f648cff5321d4ac9f3cb2" exitCode=0 Jul 14 11:36:28.961232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:28.960889 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" event={"ID":"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94","Type":"ContainerDied","Data":"a983240c7fd2363f59fcabfda787c1a8f366017fc40f648cff5321d4ac9f3cb2"} Jul 14 11:36:29.460859 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.460827 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:36:29.524459 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.524424 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-extensions-socket-volume\") pod \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " Jul 14 11:36:29.524612 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.524507 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-kube-api-access-g8ftp\") pod \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\" (UID: \"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94\") " Jul 14 11:36:29.524781 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.524757 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" (UID: "3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:36:29.526747 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.526722 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-kube-api-access-g8ftp" (OuterVolumeSpecName: "kube-api-access-g8ftp") pod "3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" (UID: "3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94"). InnerVolumeSpecName "kube-api-access-g8ftp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:36:29.625906 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.625825 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-extensions-socket-volume\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:36:29.625906 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.625854 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94-kube-api-access-g8ftp\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:36:29.968282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.968174 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" Jul 14 11:36:29.968282 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.968188 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd" event={"ID":"3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94","Type":"ContainerDied","Data":"f0613bed8277c6141ef809f9f19508c30d44f7e5d679cc4436185bfac18f2152"} Jul 14 11:36:29.968714 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.968307 2568 scope.go:117] "RemoveContainer" containerID="a983240c7fd2363f59fcabfda787c1a8f366017fc40f648cff5321d4ac9f3cb2" Jul 14 11:36:29.993488 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.993458 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd"] Jul 14 11:36:29.997780 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:29.997759 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7xtzd"] Jul 14 11:36:30.678342 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:36:30.678309 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" path="/var/lib/kubelet/pods/3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94/volumes" Jul 14 11:37:27.424337 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424303 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5"] Jul 14 11:37:27.424788 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424711 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb408680-76f3-432e-b2e7-a2c1a47c40ca" containerName="cleanup" Jul 14 11:37:27.424788 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424722 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb408680-76f3-432e-b2e7-a2c1a47c40ca" containerName="cleanup" Jul 14 11:37:27.424788 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424737 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" containerName="manager" Jul 14 11:37:27.424788 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424745 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" containerName="manager" Jul 14 11:37:27.424913 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424833 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb408680-76f3-432e-b2e7-a2c1a47c40ca" containerName="cleanup" Jul 14 11:37:27.424913 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.424843 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd75c9a-29ec-482a-b6d0-6d32a2ab2a94" containerName="manager" Jul 14 11:37:27.427770 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.427751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.431042 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.431021 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-48pbg\"" Jul 14 11:37:27.439783 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.439764 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5"] Jul 14 11:37:27.529869 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.529829 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47sl9\" (UniqueName: \"kubernetes.io/projected/8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c-kube-api-access-47sl9\") pod \"kuadrant-operator-controller-manager-86878c769f-wqkc5\" (UID: \"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.530029 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.529915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wqkc5\" (UID: \"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.630683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.630645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wqkc5\" (UID: \"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.630848 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.630716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47sl9\" (UniqueName: \"kubernetes.io/projected/8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c-kube-api-access-47sl9\") pod \"kuadrant-operator-controller-manager-86878c769f-wqkc5\" (UID: \"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.631040 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.631021 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wqkc5\" (UID: \"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.639703 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.639682 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47sl9\" (UniqueName: \"kubernetes.io/projected/8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c-kube-api-access-47sl9\") pod \"kuadrant-operator-controller-manager-86878c769f-wqkc5\" (UID: \"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.738080 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.737993 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:27.883523 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:27.883499 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5"] Jul 14 11:37:27.885687 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:37:27.885647 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8235bfdc_ea6c_4f3d_9c3f_bf98f12da26c.slice/crio-86376c55bba82807cc924d8e02c5f99b07b291f0e1641d928410882c5c99e7eb WatchSource:0}: Error finding container 86376c55bba82807cc924d8e02c5f99b07b291f0e1641d928410882c5c99e7eb: Status 404 returned error can't find the container with id 86376c55bba82807cc924d8e02c5f99b07b291f0e1641d928410882c5c99e7eb Jul 14 11:37:28.242139 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:28.242102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" event={"ID":"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c","Type":"ContainerStarted","Data":"6b6fa81d3a8f012bfbcab43ee31d89a45568db3f578ca616196d8a385f6d7195"} Jul 14 11:37:28.242339 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:28.242164 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" event={"ID":"8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c","Type":"ContainerStarted","Data":"86376c55bba82807cc924d8e02c5f99b07b291f0e1641d928410882c5c99e7eb"} Jul 14 11:37:28.242339 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:28.242228 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:28.263508 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:28.263448 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" podStartSLOduration=1.263430003 podStartE2EDuration="1.263430003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:37:28.261707403 +0000 UTC m=+1178.117541575" watchObservedRunningTime="2026-07-14 11:37:28.263430003 +0000 UTC m=+1178.119264174" Jul 14 11:37:39.248924 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:39.248889 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wqkc5" Jul 14 11:37:50.652390 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:50.652365 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:37:50.654313 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:50.654288 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:37:50.660306 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:50.660286 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:37:50.662018 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:37:50.662000 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:42:50.683768 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:42:50.683739 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:42:50.686550 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:42:50.686531 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:42:50.691522 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:42:50.691500 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:42:50.694396 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:42:50.694377 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:43:32.343101 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.343063 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh"] Jul 14 11:43:32.346580 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.346559 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:32.349818 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.349796 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\"" Jul 14 11:43:32.362271 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.362234 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh"] Jul 14 11:43:32.524290 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.524229 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d40f894-963a-44d9-ace8-cbb356a20090-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh\" (UID: \"3d40f894-963a-44d9-ace8-cbb356a20090\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:32.625004 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.624921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d40f894-963a-44d9-ace8-cbb356a20090-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh\" (UID: \"3d40f894-963a-44d9-ace8-cbb356a20090\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:32.625549 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.625530 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d40f894-963a-44d9-ace8-cbb356a20090-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh\" (UID: \"3d40f894-963a-44d9-ace8-cbb356a20090\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:32.656912 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.656890 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:32.801457 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:32.801406 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh"] Jul 14 11:43:32.803836 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:43:32.803797 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d40f894_963a_44d9_ace8_cbb356a20090.slice/crio-2fba9d2bf57e381507f3d43fa05a39d73561b3bd309f88682885f4b31a290297 WatchSource:0}: Error finding container 2fba9d2bf57e381507f3d43fa05a39d73561b3bd309f88682885f4b31a290297: Status 404 returned error can't find the container with id 2fba9d2bf57e381507f3d43fa05a39d73561b3bd309f88682885f4b31a290297 Jul 14 11:43:33.017477 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:33.017401 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" event={"ID":"3d40f894-963a-44d9-ace8-cbb356a20090","Type":"ContainerStarted","Data":"2fba9d2bf57e381507f3d43fa05a39d73561b3bd309f88682885f4b31a290297"} Jul 14 11:43:34.024492 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:34.024459 2568 generic.go:356] "Generic (PLEG): container finished" podID="3d40f894-963a-44d9-ace8-cbb356a20090" containerID="9046ec9a3d02dd75a343e4fec6a691555e2726f2925b347ccc62af60e7417b27" exitCode=7 Jul 14 11:43:34.024492 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:34.024496 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" event={"ID":"3d40f894-963a-44d9-ace8-cbb356a20090","Type":"ContainerDied","Data":"9046ec9a3d02dd75a343e4fec6a691555e2726f2925b347ccc62af60e7417b27"} Jul 14 11:43:34.024917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:34.024700 2568 scope.go:117] "RemoveContainer" containerID="9046ec9a3d02dd75a343e4fec6a691555e2726f2925b347ccc62af60e7417b27" Jul 14 11:43:35.031452 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:35.031352 2568 generic.go:356] "Generic (PLEG): container finished" podID="3d40f894-963a-44d9-ace8-cbb356a20090" containerID="d17d06918ecc94d3719db10afc8ec5f32e626a83360953f27a32ceb826eed705" exitCode=7 Jul 14 11:43:35.031452 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:35.031392 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" event={"ID":"3d40f894-963a-44d9-ace8-cbb356a20090","Type":"ContainerDied","Data":"d17d06918ecc94d3719db10afc8ec5f32e626a83360953f27a32ceb826eed705"} Jul 14 11:43:35.031452 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:35.031446 2568 scope.go:117] "RemoveContainer" containerID="9046ec9a3d02dd75a343e4fec6a691555e2726f2925b347ccc62af60e7417b27" Jul 14 11:43:35.032218 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:35.031883 2568 scope.go:117] "RemoveContainer" containerID="d17d06918ecc94d3719db10afc8ec5f32e626a83360953f27a32ceb826eed705" Jul 14 11:43:35.037975 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:43:35.032517 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh_odh-ai-gateway-infra(3d40f894-963a-44d9-ace8-cbb356a20090)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" Jul 14 11:43:36.038184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:36.038158 2568 scope.go:117] "RemoveContainer" containerID="d17d06918ecc94d3719db10afc8ec5f32e626a83360953f27a32ceb826eed705" Jul 14 11:43:36.038583 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:43:36.038399 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh_odh-ai-gateway-infra(3d40f894-963a-44d9-ace8-cbb356a20090)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" Jul 14 11:43:50.675186 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:50.675153 2568 scope.go:117] "RemoveContainer" containerID="d17d06918ecc94d3719db10afc8ec5f32e626a83360953f27a32ceb826eed705" Jul 14 11:43:51.117996 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:51.117963 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" event={"ID":"3d40f894-963a-44d9-ace8-cbb356a20090","Type":"ContainerStarted","Data":"517f740bc35cea7641da51c3e1480585c50a4055bd4e3dd87bb230b943d627ba"} Jul 14 11:43:52.125056 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:52.125020 2568 generic.go:356] "Generic (PLEG): container finished" podID="3d40f894-963a-44d9-ace8-cbb356a20090" containerID="517f740bc35cea7641da51c3e1480585c50a4055bd4e3dd87bb230b943d627ba" exitCode=0 Jul 14 11:43:52.125510 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:52.125095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" event={"ID":"3d40f894-963a-44d9-ace8-cbb356a20090","Type":"ContainerDied","Data":"517f740bc35cea7641da51c3e1480585c50a4055bd4e3dd87bb230b943d627ba"} Jul 14 11:43:52.125510 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:52.125164 2568 scope.go:117] "RemoveContainer" containerID="d17d06918ecc94d3719db10afc8ec5f32e626a83360953f27a32ceb826eed705" Jul 14 11:43:52.179434 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:52.179410 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh"] Jul 14 11:43:53.266114 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:53.266089 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:53.283108 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:53.283085 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d40f894-963a-44d9-ace8-cbb356a20090-maas-api-service-ca\") pod \"3d40f894-963a-44d9-ace8-cbb356a20090\" (UID: \"3d40f894-963a-44d9-ace8-cbb356a20090\") " Jul 14 11:43:53.283446 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:53.283421 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d40f894-963a-44d9-ace8-cbb356a20090-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d40f894-963a-44d9-ace8-cbb356a20090" (UID: "3d40f894-963a-44d9-ace8-cbb356a20090"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:43:53.384100 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:53.384016 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d40f894-963a-44d9-ace8-cbb356a20090-maas-api-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:43:54.137711 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:54.137674 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" Jul 14 11:43:54.137908 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:54.137714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh" event={"ID":"3d40f894-963a-44d9-ace8-cbb356a20090","Type":"ContainerDied","Data":"2fba9d2bf57e381507f3d43fa05a39d73561b3bd309f88682885f4b31a290297"} Jul 14 11:43:54.137908 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:54.137797 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fba9d2bf57e381507f3d43fa05a39d73561b3bd309f88682885f4b31a290297" Jul 14 11:43:54.183201 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:54.183171 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh"] Jul 14 11:43:54.190402 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:54.190377 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-7kcfh"] Jul 14 11:43:54.679154 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:43:54.679107 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" path="/var/lib/kubelet/pods/3d40f894-963a-44d9-ace8-cbb356a20090/volumes" Jul 14 11:44:32.417345 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417307 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn"] Jul 14 11:44:32.417838 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417742 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:32.417838 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417754 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:32.417838 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417770 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:32.417838 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417775 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:32.417985 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417861 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:32.417985 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.417873 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:32.420969 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.420952 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:32.423777 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.423750 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\"" Jul 14 11:44:32.429287 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.428694 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn"] Jul 14 11:44:32.531825 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.531784 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf326e7c-575a-4bc8-939c-0f139b290745-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn\" (UID: \"cf326e7c-575a-4bc8-939c-0f139b290745\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:32.632423 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.632389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf326e7c-575a-4bc8-939c-0f139b290745-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn\" (UID: \"cf326e7c-575a-4bc8-939c-0f139b290745\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:32.632968 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.632948 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf326e7c-575a-4bc8-939c-0f139b290745-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn\" (UID: \"cf326e7c-575a-4bc8-939c-0f139b290745\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:32.732682 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.732595 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:32.866293 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:32.866247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn"] Jul 14 11:44:32.868756 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:44:32.868715 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf326e7c_575a_4bc8_939c_0f139b290745.slice/crio-35fc53b099bf3e7663a6ff580a03183c978da51fca10bb4b4ebc5d020dedbe25 WatchSource:0}: Error finding container 35fc53b099bf3e7663a6ff580a03183c978da51fca10bb4b4ebc5d020dedbe25: Status 404 returned error can't find the container with id 35fc53b099bf3e7663a6ff580a03183c978da51fca10bb4b4ebc5d020dedbe25 Jul 14 11:44:33.328466 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:33.328426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" event={"ID":"cf326e7c-575a-4bc8-939c-0f139b290745","Type":"ContainerStarted","Data":"d6bcb1b20eaa015429a8b968d39228d932e1c61a535b5758df8ac70d713eec3f"} Jul 14 11:44:33.328633 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:33.328526 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" event={"ID":"cf326e7c-575a-4bc8-939c-0f139b290745","Type":"ContainerStarted","Data":"35fc53b099bf3e7663a6ff580a03183c978da51fca10bb4b4ebc5d020dedbe25"} Jul 14 11:44:33.345714 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:33.345667 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" podStartSLOduration=1.3456532860000001 podStartE2EDuration="1.345653286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:44:33.345356045 +0000 UTC m=+1603.201190218" watchObservedRunningTime="2026-07-14 11:44:33.345653286 +0000 UTC m=+1603.201487457" Jul 14 11:44:34.335039 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:34.335006 2568 generic.go:356] "Generic (PLEG): container finished" podID="cf326e7c-575a-4bc8-939c-0f139b290745" containerID="d6bcb1b20eaa015429a8b968d39228d932e1c61a535b5758df8ac70d713eec3f" exitCode=0 Jul 14 11:44:34.335429 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:34.335096 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" event={"ID":"cf326e7c-575a-4bc8-939c-0f139b290745","Type":"ContainerDied","Data":"d6bcb1b20eaa015429a8b968d39228d932e1c61a535b5758df8ac70d713eec3f"} Jul 14 11:44:35.477538 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:35.477513 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:35.661127 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:35.661039 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf326e7c-575a-4bc8-939c-0f139b290745-maas-api-service-ca\") pod \"cf326e7c-575a-4bc8-939c-0f139b290745\" (UID: \"cf326e7c-575a-4bc8-939c-0f139b290745\") " Jul 14 11:44:35.661438 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:35.661415 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf326e7c-575a-4bc8-939c-0f139b290745-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf326e7c-575a-4bc8-939c-0f139b290745" (UID: "cf326e7c-575a-4bc8-939c-0f139b290745"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:44:35.762184 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:35.762149 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf326e7c-575a-4bc8-939c-0f139b290745-maas-api-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:44:36.347327 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:36.347224 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" Jul 14 11:44:36.347327 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:36.347221 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn" event={"ID":"cf326e7c-575a-4bc8-939c-0f139b290745","Type":"ContainerDied","Data":"35fc53b099bf3e7663a6ff580a03183c978da51fca10bb4b4ebc5d020dedbe25"} Jul 14 11:44:36.347327 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:36.347315 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35fc53b099bf3e7663a6ff580a03183c978da51fca10bb4b4ebc5d020dedbe25" Jul 14 11:44:59.760852 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.760814 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw"] Jul 14 11:44:59.762230 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.762203 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:59.762423 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.762411 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:59.762540 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.762531 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf326e7c-575a-4bc8-939c-0f139b290745" containerName="revoke-keys" Jul 14 11:44:59.762606 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.762598 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf326e7c-575a-4bc8-939c-0f139b290745" containerName="revoke-keys" Jul 14 11:44:59.762877 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.762863 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d40f894-963a-44d9-ace8-cbb356a20090" containerName="revoke-keys" Jul 14 11:44:59.762977 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.762968 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf326e7c-575a-4bc8-939c-0f139b290745" containerName="revoke-keys" Jul 14 11:44:59.767077 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.767045 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.771119 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.771098 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-f22ceb3a-serving-cert\"" Jul 14 11:44:59.771234 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.771105 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:44:59.777561 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.777539 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw"] Jul 14 11:44:59.866423 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.866376 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7n82\" (UniqueName: \"kubernetes.io/projected/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-kube-api-access-x7n82\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.866423 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.866419 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.866663 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.866548 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-openshift-service-ca\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.967767 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.967724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-openshift-service-ca\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.967943 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.967777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7n82\" (UniqueName: \"kubernetes.io/projected/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-kube-api-access-x7n82\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.967943 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.967910 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.968058 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:44:59.968041 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-serving-cert: secret "maas-api-e2e-migrate-f22ceb3a-serving-cert" not found Jul 14 11:44:59.968137 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:44:59.968126 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls podName:4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a nodeName:}" failed. No retries permitted until 2026-07-14 11:45:00.468110604 +0000 UTC m=+1630.323944754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls") pod "maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" (UID: "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a") : secret "maas-api-e2e-migrate-f22ceb3a-serving-cert" not found Jul 14 11:44:59.968405 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.968386 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-openshift-service-ca\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:44:59.979644 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:44:59.979622 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7n82\" (UniqueName: \"kubernetes.io/projected/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-kube-api-access-x7n82\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:45:00.473102 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:00.473067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:45:00.475754 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:00.475729 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls\") pod \"maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:45:00.679768 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:00.679741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:45:00.818000 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:00.817970 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw"] Jul 14 11:45:00.819552 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:45:00.819525 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4baf7b13_1f86_4537_8bbd_8ebce3ae4a4a.slice/crio-ebb43bdbf7fe130fb641013eba3afd6ad33d72c32ef057367b07f31f1e7ce24d WatchSource:0}: Error finding container ebb43bdbf7fe130fb641013eba3afd6ad33d72c32ef057367b07f31f1e7ce24d: Status 404 returned error can't find the container with id ebb43bdbf7fe130fb641013eba3afd6ad33d72c32ef057367b07f31f1e7ce24d Jul 14 11:45:00.820898 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:00.820878 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:45:01.479175 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:01.479137 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" event={"ID":"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a","Type":"ContainerStarted","Data":"ebb43bdbf7fe130fb641013eba3afd6ad33d72c32ef057367b07f31f1e7ce24d"} Jul 14 11:45:03.490896 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:03.490859 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" event={"ID":"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a","Type":"ContainerStarted","Data":"ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792"} Jul 14 11:45:03.491363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:03.490920 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:45:03.513786 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:03.513734 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" podStartSLOduration=2.312885917 podStartE2EDuration="4.513713743s" totalImagesPullingTime="2.200827826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:45:03.511151815 +0000 UTC m=+1633.366985987" watchObservedRunningTime="2026-07-14 11:45:03.513713743 +0000 UTC m=+1633.369547915" Jul 14 11:45:09.500520 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:45:09.500491 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:46:10.058201 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.058131 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw"] Jul 14 11:46:10.058983 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.058949 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" podUID="4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" containerName="maas-api" containerID="cri-o://ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792" gracePeriod=30 Jul 14 11:46:10.314897 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.314833 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:46:10.359496 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.359469 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls\") pod \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " Jul 14 11:46:10.359657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.359527 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7n82\" (UniqueName: \"kubernetes.io/projected/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-kube-api-access-x7n82\") pod \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " Jul 14 11:46:10.359657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.359563 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-openshift-service-ca\") pod \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\" (UID: \"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a\") " Jul 14 11:46:10.359980 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.359954 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" (UID: "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:46:10.361769 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.361742 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" (UID: "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:46:10.361769 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.361743 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-kube-api-access-x7n82" (OuterVolumeSpecName: "kube-api-access-x7n82") pod "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" (UID: "4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a"). InnerVolumeSpecName "kube-api-access-x7n82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:46:10.460570 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.460540 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:46:10.460570 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.460568 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7n82\" (UniqueName: \"kubernetes.io/projected/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-kube-api-access-x7n82\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:46:10.460570 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.460577 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:46:10.826017 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.825984 2568 generic.go:356] "Generic (PLEG): container finished" podID="4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" containerID="ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792" exitCode=0 Jul 14 11:46:10.826221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.826027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" event={"ID":"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a","Type":"ContainerDied","Data":"ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792"} Jul 14 11:46:10.826221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.826070 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" event={"ID":"4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a","Type":"ContainerDied","Data":"ebb43bdbf7fe130fb641013eba3afd6ad33d72c32ef057367b07f31f1e7ce24d"} Jul 14 11:46:10.826221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.826091 2568 scope.go:117] "RemoveContainer" containerID="ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792" Jul 14 11:46:10.826221 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.826031 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw" Jul 14 11:46:10.836459 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.836437 2568 scope.go:117] "RemoveContainer" containerID="ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792" Jul 14 11:46:10.836737 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:46:10.836719 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792\": container with ID starting with ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792 not found: ID does not exist" containerID="ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792" Jul 14 11:46:10.836817 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.836751 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792"} err="failed to get container status \"ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792\": rpc error: code = NotFound desc = could not find container \"ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792\": container with ID starting with ee74c25d0537092d1748dd77b4f6a21bf0fc7eeb3ca64e7054d80025e3bca792 not found: ID does not exist" Jul 14 11:46:10.846185 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.846161 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw"] Jul 14 11:46:10.850075 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:10.850053 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f22ceb3a-58677744cd-z89pw"] Jul 14 11:46:12.679164 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:12.679070 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" path="/var/lib/kubelet/pods/4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a/volumes" Jul 14 11:46:17.969194 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.969165 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg"] Jul 14 11:46:17.969746 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.969718 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" containerName="maas-api" Jul 14 11:46:17.969746 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.969747 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" containerName="maas-api" Jul 14 11:46:17.969955 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.969865 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4baf7b13-1f86-4537-8bbd-8ebce3ae4a4a" containerName="maas-api" Jul 14 11:46:17.974209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.974190 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:17.977092 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.977069 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4b103af8-serving-cert\"" Jul 14 11:46:17.977215 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.977158 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:46:17.984772 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:17.984744 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg"] Jul 14 11:46:18.131747 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.131712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e210c5e-c790-4eba-ac9f-9804470221e0-maas-api-tls\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.131914 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.131779 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e210c5e-c790-4eba-ac9f-9804470221e0-openshift-service-ca\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.131914 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.131797 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlk56\" (UniqueName: \"kubernetes.io/projected/0e210c5e-c790-4eba-ac9f-9804470221e0-kube-api-access-xlk56\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.232694 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.232604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e210c5e-c790-4eba-ac9f-9804470221e0-openshift-service-ca\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.232694 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.232638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlk56\" (UniqueName: \"kubernetes.io/projected/0e210c5e-c790-4eba-ac9f-9804470221e0-kube-api-access-xlk56\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.232912 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.232771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e210c5e-c790-4eba-ac9f-9804470221e0-maas-api-tls\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.233370 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.233348 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e210c5e-c790-4eba-ac9f-9804470221e0-openshift-service-ca\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.235286 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.235248 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e210c5e-c790-4eba-ac9f-9804470221e0-maas-api-tls\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.244235 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.244211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlk56\" (UniqueName: \"kubernetes.io/projected/0e210c5e-c790-4eba-ac9f-9804470221e0-kube-api-access-xlk56\") pod \"maas-api-e2e-ait-4b103af8-749847cf8-7lzdg\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.286789 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.286752 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:18.427360 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.427335 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg"] Jul 14 11:46:18.429572 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:46:18.429530 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e210c5e_c790_4eba_ac9f_9804470221e0.slice/crio-3d1d29145e2a796e3ec1f2202f79d56d53df981bb77c9f4569a59808b917d54f WatchSource:0}: Error finding container 3d1d29145e2a796e3ec1f2202f79d56d53df981bb77c9f4569a59808b917d54f: Status 404 returned error can't find the container with id 3d1d29145e2a796e3ec1f2202f79d56d53df981bb77c9f4569a59808b917d54f Jul 14 11:46:18.870430 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:18.870395 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" event={"ID":"0e210c5e-c790-4eba-ac9f-9804470221e0","Type":"ContainerStarted","Data":"3d1d29145e2a796e3ec1f2202f79d56d53df981bb77c9f4569a59808b917d54f"} Jul 14 11:46:19.877068 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:19.877024 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" event={"ID":"0e210c5e-c790-4eba-ac9f-9804470221e0","Type":"ContainerStarted","Data":"ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9"} Jul 14 11:46:19.877472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:19.877109 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:46:19.920593 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:19.920543 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" podStartSLOduration=2.510635331 podStartE2EDuration="2.920525224s" totalImagesPullingTime="409.889893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:19.918935917 +0000 UTC m=+1709.774770090" watchObservedRunningTime="2026-07-14 11:46:19.920525224 +0000 UTC m=+1709.776359396" Jul 14 11:46:25.887988 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:46:25.887958 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:47:23.518907 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.518869 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg"] Jul 14 11:47:23.519811 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.519126 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" podUID="0e210c5e-c790-4eba-ac9f-9804470221e0" containerName="maas-api" containerID="cri-o://ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9" gracePeriod=30 Jul 14 11:47:23.767809 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.767782 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:47:23.899931 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.899900 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e210c5e-c790-4eba-ac9f-9804470221e0-openshift-service-ca\") pod \"0e210c5e-c790-4eba-ac9f-9804470221e0\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " Jul 14 11:47:23.900114 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.899969 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e210c5e-c790-4eba-ac9f-9804470221e0-maas-api-tls\") pod \"0e210c5e-c790-4eba-ac9f-9804470221e0\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " Jul 14 11:47:23.900114 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.900082 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlk56\" (UniqueName: \"kubernetes.io/projected/0e210c5e-c790-4eba-ac9f-9804470221e0-kube-api-access-xlk56\") pod \"0e210c5e-c790-4eba-ac9f-9804470221e0\" (UID: \"0e210c5e-c790-4eba-ac9f-9804470221e0\") " Jul 14 11:47:23.900324 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.900291 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e210c5e-c790-4eba-ac9f-9804470221e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e210c5e-c790-4eba-ac9f-9804470221e0" (UID: "0e210c5e-c790-4eba-ac9f-9804470221e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:47:23.900444 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.900414 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e210c5e-c790-4eba-ac9f-9804470221e0-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:47:23.902358 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.902335 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e210c5e-c790-4eba-ac9f-9804470221e0-kube-api-access-xlk56" (OuterVolumeSpecName: "kube-api-access-xlk56") pod "0e210c5e-c790-4eba-ac9f-9804470221e0" (UID: "0e210c5e-c790-4eba-ac9f-9804470221e0"). InnerVolumeSpecName "kube-api-access-xlk56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:47:23.902450 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:23.902377 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e210c5e-c790-4eba-ac9f-9804470221e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e210c5e-c790-4eba-ac9f-9804470221e0" (UID: "0e210c5e-c790-4eba-ac9f-9804470221e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:47:24.001811 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.001776 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlk56\" (UniqueName: \"kubernetes.io/projected/0e210c5e-c790-4eba-ac9f-9804470221e0-kube-api-access-xlk56\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:47:24.001811 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.001803 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e210c5e-c790-4eba-ac9f-9804470221e0-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:47:24.213850 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.213749 2568 generic.go:356] "Generic (PLEG): container finished" podID="0e210c5e-c790-4eba-ac9f-9804470221e0" containerID="ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9" exitCode=0 Jul 14 11:47:24.213850 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.213805 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" Jul 14 11:47:24.213850 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.213835 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" event={"ID":"0e210c5e-c790-4eba-ac9f-9804470221e0","Type":"ContainerDied","Data":"ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9"} Jul 14 11:47:24.214081 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.213896 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg" event={"ID":"0e210c5e-c790-4eba-ac9f-9804470221e0","Type":"ContainerDied","Data":"3d1d29145e2a796e3ec1f2202f79d56d53df981bb77c9f4569a59808b917d54f"} Jul 14 11:47:24.214081 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.213919 2568 scope.go:117] "RemoveContainer" containerID="ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9" Jul 14 11:47:24.223551 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.223536 2568 scope.go:117] "RemoveContainer" containerID="ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9" Jul 14 11:47:24.223807 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:47:24.223790 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9\": container with ID starting with ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9 not found: ID does not exist" containerID="ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9" Jul 14 11:47:24.223866 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.223814 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9"} err="failed to get container status \"ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9\": rpc error: code = NotFound desc = could not find container \"ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9\": container with ID starting with ce44a2287e2567116a06993cfde188f4a2565c1c7671ffda06a74ab49353b9a9 not found: ID does not exist" Jul 14 11:47:24.235750 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.235725 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg"] Jul 14 11:47:24.240332 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.240311 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b103af8-749847cf8-7lzdg"] Jul 14 11:47:24.678348 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:24.678318 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e210c5e-c790-4eba-ac9f-9804470221e0" path="/var/lib/kubelet/pods/0e210c5e-c790-4eba-ac9f-9804470221e0/volumes" Jul 14 11:47:40.735474 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.735435 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k"] Jul 14 11:47:40.735917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.735821 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e210c5e-c790-4eba-ac9f-9804470221e0" containerName="maas-api" Jul 14 11:47:40.735917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.735832 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e210c5e-c790-4eba-ac9f-9804470221e0" containerName="maas-api" Jul 14 11:47:40.735917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.735904 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e210c5e-c790-4eba-ac9f-9804470221e0" containerName="maas-api" Jul 14 11:47:40.738873 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.738853 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.742490 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.742464 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:47:40.743096 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.743072 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b8cbc777-serving-cert\"" Jul 14 11:47:40.751049 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.751029 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k"] Jul 14 11:47:40.849195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.849150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5l95\" (UniqueName: \"kubernetes.io/projected/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-kube-api-access-d5l95\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.849195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.849201 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.849450 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.849284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-openshift-service-ca\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.949725 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.949688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-openshift-service-ca\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.949927 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.949760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5l95\" (UniqueName: \"kubernetes.io/projected/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-kube-api-access-d5l95\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.949927 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.949800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.949927 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:47:40.949879 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-serving-cert: secret "maas-api-e2e-derive-b8cbc777-serving-cert" not found Jul 14 11:47:40.950125 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:47:40.949956 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls podName:5dd30c8b-9902-41e3-86cf-2cddfd2734f6 nodeName:}" failed. No retries permitted until 2026-07-14 11:47:41.449939975 +0000 UTC m=+1791.305774124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls") pod "maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" (UID: "5dd30c8b-9902-41e3-86cf-2cddfd2734f6") : secret "maas-api-e2e-derive-b8cbc777-serving-cert" not found Jul 14 11:47:40.950483 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.950460 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-openshift-service-ca\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:40.959200 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:40.959174 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5l95\" (UniqueName: \"kubernetes.io/projected/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-kube-api-access-d5l95\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:41.454035 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:41.453999 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:41.456516 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:41.456493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls\") pod \"maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:41.652734 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:41.652664 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:41.793786 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:41.793761 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k"] Jul 14 11:47:41.796138 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:47:41.796109 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd30c8b_9902_41e3_86cf_2cddfd2734f6.slice/crio-52e3777dbafb2a016d1e25e6143ea75ae8986f0f6e4fbefdfaa2cf6e7aafa093 WatchSource:0}: Error finding container 52e3777dbafb2a016d1e25e6143ea75ae8986f0f6e4fbefdfaa2cf6e7aafa093: Status 404 returned error can't find the container with id 52e3777dbafb2a016d1e25e6143ea75ae8986f0f6e4fbefdfaa2cf6e7aafa093 Jul 14 11:47:42.311194 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:42.311154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" event={"ID":"5dd30c8b-9902-41e3-86cf-2cddfd2734f6","Type":"ContainerStarted","Data":"52e3777dbafb2a016d1e25e6143ea75ae8986f0f6e4fbefdfaa2cf6e7aafa093"} Jul 14 11:47:43.317675 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:43.317636 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" event={"ID":"5dd30c8b-9902-41e3-86cf-2cddfd2734f6","Type":"ContainerStarted","Data":"1c0b31577e068b98522bc6509d2bc659acc8e18847a31eefed9b93269dbf4d04"} Jul 14 11:47:43.318121 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:43.317761 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:43.348829 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:43.348784 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" podStartSLOduration=2.780473163 podStartE2EDuration="3.348769001s" totalImagesPullingTime="568.295838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:47:43.347292545 +0000 UTC m=+1793.203126717" watchObservedRunningTime="2026-07-14 11:47:43.348769001 +0000 UTC m=+1793.204603213" Jul 14 11:47:49.328118 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:49.328091 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:47:50.714246 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:50.714124 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:47:50.741000 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:50.726022 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:47:50.741000 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:50.727176 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:47:50.741000 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:47:50.732689 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:48:55.422390 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.422339 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k"] Jul 14 11:48:55.423292 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.423200 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" podUID="5dd30c8b-9902-41e3-86cf-2cddfd2734f6" containerName="maas-api" containerID="cri-o://1c0b31577e068b98522bc6509d2bc659acc8e18847a31eefed9b93269dbf4d04" gracePeriod=30 Jul 14 11:48:55.674658 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.674588 2568 generic.go:356] "Generic (PLEG): container finished" podID="5dd30c8b-9902-41e3-86cf-2cddfd2734f6" containerID="1c0b31577e068b98522bc6509d2bc659acc8e18847a31eefed9b93269dbf4d04" exitCode=0 Jul 14 11:48:55.674658 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.674619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" event={"ID":"5dd30c8b-9902-41e3-86cf-2cddfd2734f6","Type":"ContainerDied","Data":"1c0b31577e068b98522bc6509d2bc659acc8e18847a31eefed9b93269dbf4d04"} Jul 14 11:48:55.674812 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.674671 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" event={"ID":"5dd30c8b-9902-41e3-86cf-2cddfd2734f6","Type":"ContainerDied","Data":"52e3777dbafb2a016d1e25e6143ea75ae8986f0f6e4fbefdfaa2cf6e7aafa093"} Jul 14 11:48:55.674812 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.674688 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52e3777dbafb2a016d1e25e6143ea75ae8986f0f6e4fbefdfaa2cf6e7aafa093" Jul 14 11:48:55.680531 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.680512 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:48:55.778284 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.778240 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls\") pod \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " Jul 14 11:48:55.778478 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.778350 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-openshift-service-ca\") pod \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " Jul 14 11:48:55.778478 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.778374 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5l95\" (UniqueName: \"kubernetes.io/projected/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-kube-api-access-d5l95\") pod \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\" (UID: \"5dd30c8b-9902-41e3-86cf-2cddfd2734f6\") " Jul 14 11:48:55.778712 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.778685 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dd30c8b-9902-41e3-86cf-2cddfd2734f6" (UID: "5dd30c8b-9902-41e3-86cf-2cddfd2734f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:48:55.780549 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.780524 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-kube-api-access-d5l95" (OuterVolumeSpecName: "kube-api-access-d5l95") pod "5dd30c8b-9902-41e3-86cf-2cddfd2734f6" (UID: "5dd30c8b-9902-41e3-86cf-2cddfd2734f6"). InnerVolumeSpecName "kube-api-access-d5l95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:48:55.780650 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.780592 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dd30c8b-9902-41e3-86cf-2cddfd2734f6" (UID: "5dd30c8b-9902-41e3-86cf-2cddfd2734f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:48:55.879870 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.879838 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:48:55.879870 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.879865 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:48:55.879870 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:55.879875 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5l95\" (UniqueName: \"kubernetes.io/projected/5dd30c8b-9902-41e3-86cf-2cddfd2734f6-kube-api-access-d5l95\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:48:56.679161 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:56.679132 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k" Jul 14 11:48:56.701330 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:56.701305 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k"] Jul 14 11:48:56.705161 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:56.705137 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cbc777-986dbc8c9-mdk4k"] Jul 14 11:48:58.679843 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:48:58.679808 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd30c8b-9902-41e3-86cf-2cddfd2734f6" path="/var/lib/kubelet/pods/5dd30c8b-9902-41e3-86cf-2cddfd2734f6/volumes" Jul 14 11:49:50.818611 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:49:50.818580 2568 scope.go:117] "RemoveContainer" containerID="517f740bc35cea7641da51c3e1480585c50a4055bd4e3dd87bb230b943d627ba" Jul 14 11:50:50.344883 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.344807 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg"] Jul 14 11:50:50.345378 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.345197 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd30c8b-9902-41e3-86cf-2cddfd2734f6" containerName="maas-api" Jul 14 11:50:50.345378 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.345209 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd30c8b-9902-41e3-86cf-2cddfd2734f6" containerName="maas-api" Jul 14 11:50:50.345378 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.345290 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd30c8b-9902-41e3-86cf-2cddfd2734f6" containerName="maas-api" Jul 14 11:50:50.348479 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.348463 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.351477 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.351453 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3f0f93e1-serving-cert\"" Jul 14 11:50:50.351599 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.351545 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:50:50.358852 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.358829 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg"] Jul 14 11:50:50.374852 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.374822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4079f33-6d31-4bda-9454-ff92e5b8e422-openshift-service-ca\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.374978 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.374862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5llr\" (UniqueName: \"kubernetes.io/projected/c4079f33-6d31-4bda-9454-ff92e5b8e422-kube-api-access-z5llr\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.374978 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.374953 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.475749 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.475706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.475929 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.475827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4079f33-6d31-4bda-9454-ff92e5b8e422-openshift-service-ca\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.475929 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:50:50.475850 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-serving-cert: secret "maas-api-e2e-mt-3f0f93e1-serving-cert" not found Jul 14 11:50:50.475929 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.475858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5llr\" (UniqueName: \"kubernetes.io/projected/c4079f33-6d31-4bda-9454-ff92e5b8e422-kube-api-access-z5llr\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.475929 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:50:50.475928 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls podName:c4079f33-6d31-4bda-9454-ff92e5b8e422 nodeName:}" failed. No retries permitted until 2026-07-14 11:50:50.975906279 +0000 UTC m=+1980.831740429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls") pod "maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" (UID: "c4079f33-6d31-4bda-9454-ff92e5b8e422") : secret "maas-api-e2e-mt-3f0f93e1-serving-cert" not found Jul 14 11:50:50.476483 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.476463 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4079f33-6d31-4bda-9454-ff92e5b8e422-openshift-service-ca\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.489496 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.489471 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5llr\" (UniqueName: \"kubernetes.io/projected/c4079f33-6d31-4bda-9454-ff92e5b8e422-kube-api-access-z5llr\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.979979 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:50.979944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:50.980152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:50:50.980063 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-serving-cert: secret "maas-api-e2e-mt-3f0f93e1-serving-cert" not found Jul 14 11:50:50.980152 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:50:50.980114 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls podName:c4079f33-6d31-4bda-9454-ff92e5b8e422 nodeName:}" failed. No retries permitted until 2026-07-14 11:50:51.980100841 +0000 UTC m=+1981.835934992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls") pod "maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" (UID: "c4079f33-6d31-4bda-9454-ff92e5b8e422") : secret "maas-api-e2e-mt-3f0f93e1-serving-cert" not found Jul 14 11:50:51.987297 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:51.987229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:51.989919 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:51.989891 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") pod \"maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:52.162670 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:52.162636 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:50:52.170363 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:52.170344 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:52.315542 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:52.315513 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg"] Jul 14 11:50:52.318129 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:50:52.318084 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4079f33_6d31_4bda_9454_ff92e5b8e422.slice/crio-737c1225294e16f08558bcffafb68fe217c601884eed87c70985f568eadd3993 WatchSource:0}: Error finding container 737c1225294e16f08558bcffafb68fe217c601884eed87c70985f568eadd3993: Status 404 returned error can't find the container with id 737c1225294e16f08558bcffafb68fe217c601884eed87c70985f568eadd3993 Jul 14 11:50:52.319555 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:52.319535 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:50:53.272417 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:53.272318 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" event={"ID":"c4079f33-6d31-4bda-9454-ff92e5b8e422","Type":"ContainerStarted","Data":"5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c"} Jul 14 11:50:53.272417 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:53.272374 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" event={"ID":"c4079f33-6d31-4bda-9454-ff92e5b8e422","Type":"ContainerStarted","Data":"737c1225294e16f08558bcffafb68fe217c601884eed87c70985f568eadd3993"} Jul 14 11:50:53.272815 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:53.272440 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:50:53.292015 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:53.291954 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" podStartSLOduration=2.802566743 podStartE2EDuration="3.291934333s" totalImagesPullingTime="489.36759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:50:53.291398257 +0000 UTC m=+1983.147232430" watchObservedRunningTime="2026-07-14 11:50:53.291934333 +0000 UTC m=+1983.147768506" Jul 14 11:50:59.283864 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:50:59.283831 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:51:12.814864 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:12.814822 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg"] Jul 14 11:51:12.815559 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:12.815072 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" podUID="c4079f33-6d31-4bda-9454-ff92e5b8e422" containerName="maas-api" containerID="cri-o://5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c" gracePeriod=30 Jul 14 11:51:13.075981 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.075919 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:51:13.179561 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.179521 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5llr\" (UniqueName: \"kubernetes.io/projected/c4079f33-6d31-4bda-9454-ff92e5b8e422-kube-api-access-z5llr\") pod \"c4079f33-6d31-4bda-9454-ff92e5b8e422\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " Jul 14 11:51:13.179746 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.179595 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4079f33-6d31-4bda-9454-ff92e5b8e422-openshift-service-ca\") pod \"c4079f33-6d31-4bda-9454-ff92e5b8e422\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " Jul 14 11:51:13.179746 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.179636 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") pod \"c4079f33-6d31-4bda-9454-ff92e5b8e422\" (UID: \"c4079f33-6d31-4bda-9454-ff92e5b8e422\") " Jul 14 11:51:13.179997 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.179966 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4079f33-6d31-4bda-9454-ff92e5b8e422-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4079f33-6d31-4bda-9454-ff92e5b8e422" (UID: "c4079f33-6d31-4bda-9454-ff92e5b8e422"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:51:13.181865 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.181839 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4079f33-6d31-4bda-9454-ff92e5b8e422" (UID: "c4079f33-6d31-4bda-9454-ff92e5b8e422"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:51:13.181970 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.181896 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4079f33-6d31-4bda-9454-ff92e5b8e422-kube-api-access-z5llr" (OuterVolumeSpecName: "kube-api-access-z5llr") pod "c4079f33-6d31-4bda-9454-ff92e5b8e422" (UID: "c4079f33-6d31-4bda-9454-ff92e5b8e422"). InnerVolumeSpecName "kube-api-access-z5llr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:51:13.280832 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.280797 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4079f33-6d31-4bda-9454-ff92e5b8e422-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:51:13.280832 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.280828 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4079f33-6d31-4bda-9454-ff92e5b8e422-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:51:13.280832 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.280838 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5llr\" (UniqueName: \"kubernetes.io/projected/c4079f33-6d31-4bda-9454-ff92e5b8e422-kube-api-access-z5llr\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:51:13.374764 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.374675 2568 generic.go:356] "Generic (PLEG): container finished" podID="c4079f33-6d31-4bda-9454-ff92e5b8e422" containerID="5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c" exitCode=0 Jul 14 11:51:13.374764 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.374724 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" Jul 14 11:51:13.374945 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.374762 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" event={"ID":"c4079f33-6d31-4bda-9454-ff92e5b8e422","Type":"ContainerDied","Data":"5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c"} Jul 14 11:51:13.374945 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.374823 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg" event={"ID":"c4079f33-6d31-4bda-9454-ff92e5b8e422","Type":"ContainerDied","Data":"737c1225294e16f08558bcffafb68fe217c601884eed87c70985f568eadd3993"} Jul 14 11:51:13.374945 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.374846 2568 scope.go:117] "RemoveContainer" containerID="5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c" Jul 14 11:51:13.384035 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.384015 2568 scope.go:117] "RemoveContainer" containerID="5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c" Jul 14 11:51:13.384334 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:51:13.384313 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c\": container with ID starting with 5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c not found: ID does not exist" containerID="5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c" Jul 14 11:51:13.384388 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.384344 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c"} err="failed to get container status \"5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c\": rpc error: code = NotFound desc = could not find container \"5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c\": container with ID starting with 5c774aee2d9c0dcfd7ee99e370e678aa2c264655044ac16a7f2c91cda2ae485c not found: ID does not exist" Jul 14 11:51:13.397662 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.397637 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg"] Jul 14 11:51:13.403006 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:13.402985 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f0f93e1-78558bd7dc-qnbwg"] Jul 14 11:51:14.678366 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:14.678333 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4079f33-6d31-4bda-9454-ff92e5b8e422" path="/var/lib/kubelet/pods/c4079f33-6d31-4bda-9454-ff92e5b8e422/volumes" Jul 14 11:51:50.663156 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.663116 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b"] Jul 14 11:51:50.663621 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.663540 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4079f33-6d31-4bda-9454-ff92e5b8e422" containerName="maas-api" Jul 14 11:51:50.663621 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.663553 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4079f33-6d31-4bda-9454-ff92e5b8e422" containerName="maas-api" Jul 14 11:51:50.663695 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.663624 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4079f33-6d31-4bda-9454-ff92e5b8e422" containerName="maas-api" Jul 14 11:51:50.666924 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.666906 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.670787 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.670764 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-67b298da-serving-cert\"" Jul 14 11:51:50.670891 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.670774 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:51:50.688619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.688590 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b"] Jul 14 11:51:50.714025 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.713996 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26hxq\" (UniqueName: \"kubernetes.io/projected/1f9be564-9e29-4d60-8444-5b40b54f8b33-kube-api-access-26hxq\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.714186 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.714054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.714186 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.714108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9be564-9e29-4d60-8444-5b40b54f8b33-openshift-service-ca\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.815198 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.815161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.815371 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.815228 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9be564-9e29-4d60-8444-5b40b54f8b33-openshift-service-ca\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.815371 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:51:50.815321 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-serving-cert: secret "maas-api-e2e-mt-67b298da-serving-cert" not found Jul 14 11:51:50.815456 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.815328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26hxq\" (UniqueName: \"kubernetes.io/projected/1f9be564-9e29-4d60-8444-5b40b54f8b33-kube-api-access-26hxq\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.815456 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:51:50.815401 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls podName:1f9be564-9e29-4d60-8444-5b40b54f8b33 nodeName:}" failed. No retries permitted until 2026-07-14 11:51:51.315376833 +0000 UTC m=+2041.171210986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls") pod "maas-api-e2e-mt-67b298da-76c589dc59-9b86b" (UID: "1f9be564-9e29-4d60-8444-5b40b54f8b33") : secret "maas-api-e2e-mt-67b298da-serving-cert" not found Jul 14 11:51:50.816097 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.816074 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9be564-9e29-4d60-8444-5b40b54f8b33-openshift-service-ca\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:50.824808 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:50.824781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26hxq\" (UniqueName: \"kubernetes.io/projected/1f9be564-9e29-4d60-8444-5b40b54f8b33-kube-api-access-26hxq\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:51.319396 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:51.319363 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:51.322058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:51.322029 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls\") pod \"maas-api-e2e-mt-67b298da-76c589dc59-9b86b\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:51.588421 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:51.588338 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:51.942792 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:51.942764 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b"] Jul 14 11:51:51.944540 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:51:51.944515 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9be564_9e29_4d60_8444_5b40b54f8b33.slice/crio-1d43f26c49970361a68f5218150b4ee128e6f5615cfdaf8d705029699a1e82f9 WatchSource:0}: Error finding container 1d43f26c49970361a68f5218150b4ee128e6f5615cfdaf8d705029699a1e82f9: Status 404 returned error can't find the container with id 1d43f26c49970361a68f5218150b4ee128e6f5615cfdaf8d705029699a1e82f9 Jul 14 11:51:52.575596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:52.575554 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" event={"ID":"1f9be564-9e29-4d60-8444-5b40b54f8b33","Type":"ContainerStarted","Data":"0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea"} Jul 14 11:51:52.575783 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:52.575602 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" event={"ID":"1f9be564-9e29-4d60-8444-5b40b54f8b33","Type":"ContainerStarted","Data":"1d43f26c49970361a68f5218150b4ee128e6f5615cfdaf8d705029699a1e82f9"} Jul 14 11:51:52.575783 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:52.575640 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:51:52.597205 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:52.597152 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" podStartSLOduration=2.28713069 podStartE2EDuration="2.597137121s" totalImagesPullingTime="310.006431ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:51:52.595387809 +0000 UTC m=+2042.451221976" watchObservedRunningTime="2026-07-14 11:51:52.597137121 +0000 UTC m=+2042.452971306" Jul 14 11:51:58.587009 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:51:58.586969 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:52:50.750743 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:50.750629 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:52:50.756408 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:50.756385 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:52:50.758102 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:50.758083 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:52:50.763868 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:50.763848 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:52:54.275365 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.275324 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b"] Jul 14 11:52:54.275808 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.275620 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" podUID="1f9be564-9e29-4d60-8444-5b40b54f8b33" containerName="maas-api" containerID="cri-o://0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea" gracePeriod=30 Jul 14 11:52:54.541618 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.541594 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:52:54.680027 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.679995 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26hxq\" (UniqueName: \"kubernetes.io/projected/1f9be564-9e29-4d60-8444-5b40b54f8b33-kube-api-access-26hxq\") pod \"1f9be564-9e29-4d60-8444-5b40b54f8b33\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " Jul 14 11:52:54.680193 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.680043 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls\") pod \"1f9be564-9e29-4d60-8444-5b40b54f8b33\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " Jul 14 11:52:54.680193 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.680074 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9be564-9e29-4d60-8444-5b40b54f8b33-openshift-service-ca\") pod \"1f9be564-9e29-4d60-8444-5b40b54f8b33\" (UID: \"1f9be564-9e29-4d60-8444-5b40b54f8b33\") " Jul 14 11:52:54.680540 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.680515 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9be564-9e29-4d60-8444-5b40b54f8b33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f9be564-9e29-4d60-8444-5b40b54f8b33" (UID: "1f9be564-9e29-4d60-8444-5b40b54f8b33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:52:54.682247 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.682225 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f9be564-9e29-4d60-8444-5b40b54f8b33" (UID: "1f9be564-9e29-4d60-8444-5b40b54f8b33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:52:54.682392 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.682298 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9be564-9e29-4d60-8444-5b40b54f8b33-kube-api-access-26hxq" (OuterVolumeSpecName: "kube-api-access-26hxq") pod "1f9be564-9e29-4d60-8444-5b40b54f8b33" (UID: "1f9be564-9e29-4d60-8444-5b40b54f8b33"). InnerVolumeSpecName "kube-api-access-26hxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:52:54.781680 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.781593 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26hxq\" (UniqueName: \"kubernetes.io/projected/1f9be564-9e29-4d60-8444-5b40b54f8b33-kube-api-access-26hxq\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:52:54.781680 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.781626 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9be564-9e29-4d60-8444-5b40b54f8b33-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:52:54.781680 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.781637 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9be564-9e29-4d60-8444-5b40b54f8b33-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:52:54.884947 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.884911 2568 generic.go:356] "Generic (PLEG): container finished" podID="1f9be564-9e29-4d60-8444-5b40b54f8b33" containerID="0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea" exitCode=0 Jul 14 11:52:54.885110 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.884958 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" Jul 14 11:52:54.885110 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.884994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" event={"ID":"1f9be564-9e29-4d60-8444-5b40b54f8b33","Type":"ContainerDied","Data":"0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea"} Jul 14 11:52:54.885110 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.885049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b" event={"ID":"1f9be564-9e29-4d60-8444-5b40b54f8b33","Type":"ContainerDied","Data":"1d43f26c49970361a68f5218150b4ee128e6f5615cfdaf8d705029699a1e82f9"} Jul 14 11:52:54.885110 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.885078 2568 scope.go:117] "RemoveContainer" containerID="0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea" Jul 14 11:52:54.894674 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.894652 2568 scope.go:117] "RemoveContainer" containerID="0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea" Jul 14 11:52:54.894896 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:52:54.894879 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea\": container with ID starting with 0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea not found: ID does not exist" containerID="0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea" Jul 14 11:52:54.894945 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.894904 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea"} err="failed to get container status \"0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea\": rpc error: code = NotFound desc = could not find container \"0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea\": container with ID starting with 0daa03721b8a3ada0ed531b605428fedb0f8e841f24eab37f6611eda3092c1ea not found: ID does not exist" Jul 14 11:52:54.909305 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.909280 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b"] Jul 14 11:52:54.916131 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:54.916095 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67b298da-76c589dc59-9b86b"] Jul 14 11:52:56.677951 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:52:56.677919 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9be564-9e29-4d60-8444-5b40b54f8b33" path="/var/lib/kubelet/pods/1f9be564-9e29-4d60-8444-5b40b54f8b33/volumes" Jul 14 11:53:50.865231 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:53:50.865146 2568 scope.go:117] "RemoveContainer" containerID="1c0b31577e068b98522bc6509d2bc659acc8e18847a31eefed9b93269dbf4d04" Jul 14 11:54:49.720593 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.720559 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg"] Jul 14 11:54:49.721062 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.720942 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9be564-9e29-4d60-8444-5b40b54f8b33" containerName="maas-api" Jul 14 11:54:49.721062 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.720953 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9be564-9e29-4d60-8444-5b40b54f8b33" containerName="maas-api" Jul 14 11:54:49.721062 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.721034 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9be564-9e29-4d60-8444-5b40b54f8b33" containerName="maas-api" Jul 14 11:54:49.724209 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.724190 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.727574 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.727544 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8567a3-serving-cert\"" Jul 14 11:54:49.727719 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.727553 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:54:49.738443 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.738418 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg"] Jul 14 11:54:49.869139 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.869105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b10765-6bea-47d5-a930-50390be8779e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.869334 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.869163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m22l\" (UniqueName: \"kubernetes.io/projected/29b10765-6bea-47d5-a930-50390be8779e-kube-api-access-4m22l\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.869334 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.869310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.970528 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.970463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.970528 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.970531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b10765-6bea-47d5-a930-50390be8779e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.970784 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.970571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m22l\" (UniqueName: \"kubernetes.io/projected/29b10765-6bea-47d5-a930-50390be8779e-kube-api-access-4m22l\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.970784 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:54:49.970652 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-serving-cert: secret "maas-api-e2e-inf-a-8567a3-serving-cert" not found Jul 14 11:54:49.970784 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:54:49.970735 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls podName:29b10765-6bea-47d5-a930-50390be8779e nodeName:}" failed. No retries permitted until 2026-07-14 11:54:50.470712543 +0000 UTC m=+2220.326546711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls") pod "maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" (UID: "29b10765-6bea-47d5-a930-50390be8779e") : secret "maas-api-e2e-inf-a-8567a3-serving-cert" not found Jul 14 11:54:49.971243 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.971223 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b10765-6bea-47d5-a930-50390be8779e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:49.984901 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:49.984877 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m22l\" (UniqueName: \"kubernetes.io/projected/29b10765-6bea-47d5-a930-50390be8779e-kube-api-access-4m22l\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:50.475572 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:50.475535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:50.478237 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:50.478210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:50.637994 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:50.637961 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:54:50.645310 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:50.645285 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:50.793058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:50.793029 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg"] Jul 14 11:54:50.794774 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:54:50.794746 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b10765_6bea_47d5_a930_50390be8779e.slice/crio-4936d475f7a6a03195eb4bdd166dc5565ed4fdbde4d7be2fa27249085938d9a8 WatchSource:0}: Error finding container 4936d475f7a6a03195eb4bdd166dc5565ed4fdbde4d7be2fa27249085938d9a8: Status 404 returned error can't find the container with id 4936d475f7a6a03195eb4bdd166dc5565ed4fdbde4d7be2fa27249085938d9a8 Jul 14 11:54:51.470725 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:51.470634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" event={"ID":"29b10765-6bea-47d5-a930-50390be8779e","Type":"ContainerStarted","Data":"ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af"} Jul 14 11:54:51.470725 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:51.470686 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" event={"ID":"29b10765-6bea-47d5-a930-50390be8779e","Type":"ContainerStarted","Data":"4936d475f7a6a03195eb4bdd166dc5565ed4fdbde4d7be2fa27249085938d9a8"} Jul 14 11:54:51.470725 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:51.470714 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:51.488493 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:51.488442 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" podStartSLOduration=2.133141935 podStartE2EDuration="2.488427573s" totalImagesPullingTime="355.285638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:54:51.487621341 +0000 UTC m=+2221.343455513" watchObservedRunningTime="2026-07-14 11:54:51.488427573 +0000 UTC m=+2221.344261781" Jul 14 11:54:57.480969 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.480893 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:54:57.829242 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.829201 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk"] Jul 14 11:54:57.832814 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.832791 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:57.835917 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.835891 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-738ce7-serving-cert\"" Jul 14 11:54:57.844761 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.844711 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk"] Jul 14 11:54:57.944063 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.944029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:57.944239 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.944073 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:57.944239 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:57.944115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnc4f\" (UniqueName: \"kubernetes.io/projected/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-kube-api-access-tnc4f\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.045003 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.044965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.045003 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.045005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.045227 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.045048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnc4f\" (UniqueName: \"kubernetes.io/projected/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-kube-api-access-tnc4f\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.045227 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:54:58.045119 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-serving-cert: secret "maas-api-e2e-inf-b-738ce7-serving-cert" not found Jul 14 11:54:58.045227 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:54:58.045192 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls podName:40f7cf8b-0bfd-4e52-9953-9c02ab869f9b nodeName:}" failed. No retries permitted until 2026-07-14 11:54:58.545175649 +0000 UTC m=+2228.401009798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls") pod "maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" (UID: "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b") : secret "maas-api-e2e-inf-b-738ce7-serving-cert" not found Jul 14 11:54:58.045693 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.045673 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.055232 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.055210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnc4f\" (UniqueName: \"kubernetes.io/projected/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-kube-api-access-tnc4f\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.550826 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.550785 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.553467 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.553446 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls\") pod \"maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.746618 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.746578 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:58.891248 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:58.891223 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk"] Jul 14 11:54:58.893222 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:54:58.893190 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f7cf8b_0bfd_4e52_9953_9c02ab869f9b.slice/crio-cfd8a1cf9c328bdc96557f1e30663f8e1d6d8ab86f08b9a24c4d14daac3d6883 WatchSource:0}: Error finding container cfd8a1cf9c328bdc96557f1e30663f8e1d6d8ab86f08b9a24c4d14daac3d6883: Status 404 returned error can't find the container with id cfd8a1cf9c328bdc96557f1e30663f8e1d6d8ab86f08b9a24c4d14daac3d6883 Jul 14 11:54:59.517630 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:59.517597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" event={"ID":"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b","Type":"ContainerStarted","Data":"a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c"} Jul 14 11:54:59.517833 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:59.517657 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" event={"ID":"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b","Type":"ContainerStarted","Data":"cfd8a1cf9c328bdc96557f1e30663f8e1d6d8ab86f08b9a24c4d14daac3d6883"} Jul 14 11:54:59.517833 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:59.517738 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:54:59.538988 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:54:59.538925 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" podStartSLOduration=2.138755245 podStartE2EDuration="2.538905965s" totalImagesPullingTime="400.15072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:54:59.537921636 +0000 UTC m=+2229.393755805" watchObservedRunningTime="2026-07-14 11:54:59.538905965 +0000 UTC m=+2229.394740137" Jul 14 11:55:05.530120 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:55:05.530075 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:56:22.888979 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:22.888939 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg"] Jul 14 11:56:22.889590 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:22.889266 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" podUID="29b10765-6bea-47d5-a930-50390be8779e" containerName="maas-api" containerID="cri-o://ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af" gracePeriod=30 Jul 14 11:56:23.143074 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.143013 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:56:23.312610 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.312569 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b10765-6bea-47d5-a930-50390be8779e-openshift-service-ca\") pod \"29b10765-6bea-47d5-a930-50390be8779e\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " Jul 14 11:56:23.312807 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.312639 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m22l\" (UniqueName: \"kubernetes.io/projected/29b10765-6bea-47d5-a930-50390be8779e-kube-api-access-4m22l\") pod \"29b10765-6bea-47d5-a930-50390be8779e\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " Jul 14 11:56:23.312807 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.312663 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls\") pod \"29b10765-6bea-47d5-a930-50390be8779e\" (UID: \"29b10765-6bea-47d5-a930-50390be8779e\") " Jul 14 11:56:23.313060 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.313021 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b10765-6bea-47d5-a930-50390be8779e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29b10765-6bea-47d5-a930-50390be8779e" (UID: "29b10765-6bea-47d5-a930-50390be8779e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:56:23.315018 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.314986 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29b10765-6bea-47d5-a930-50390be8779e" (UID: "29b10765-6bea-47d5-a930-50390be8779e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:56:23.315143 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.315041 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b10765-6bea-47d5-a930-50390be8779e-kube-api-access-4m22l" (OuterVolumeSpecName: "kube-api-access-4m22l") pod "29b10765-6bea-47d5-a930-50390be8779e" (UID: "29b10765-6bea-47d5-a930-50390be8779e"). InnerVolumeSpecName "kube-api-access-4m22l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:56:23.414061 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.413978 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m22l\" (UniqueName: \"kubernetes.io/projected/29b10765-6bea-47d5-a930-50390be8779e-kube-api-access-4m22l\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:56:23.414061 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.414016 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29b10765-6bea-47d5-a930-50390be8779e-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:56:23.414061 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.414032 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b10765-6bea-47d5-a930-50390be8779e-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:56:23.952101 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.952063 2568 generic.go:356] "Generic (PLEG): container finished" podID="29b10765-6bea-47d5-a930-50390be8779e" containerID="ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af" exitCode=0 Jul 14 11:56:23.952505 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.952113 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" Jul 14 11:56:23.952505 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.952125 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" event={"ID":"29b10765-6bea-47d5-a930-50390be8779e","Type":"ContainerDied","Data":"ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af"} Jul 14 11:56:23.952505 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.952178 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg" event={"ID":"29b10765-6bea-47d5-a930-50390be8779e","Type":"ContainerDied","Data":"4936d475f7a6a03195eb4bdd166dc5565ed4fdbde4d7be2fa27249085938d9a8"} Jul 14 11:56:23.952505 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.952208 2568 scope.go:117] "RemoveContainer" containerID="ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af" Jul 14 11:56:23.961300 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.961281 2568 scope.go:117] "RemoveContainer" containerID="ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af" Jul 14 11:56:23.961565 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:56:23.961547 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af\": container with ID starting with ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af not found: ID does not exist" containerID="ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af" Jul 14 11:56:23.961614 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.961575 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af"} err="failed to get container status \"ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af\": rpc error: code = NotFound desc = could not find container \"ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af\": container with ID starting with ada64a2e75f4e38b806e8d98a64fcc0ebe6c739a5b84692c9098e57e604425af not found: ID does not exist" Jul 14 11:56:23.977031 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.977005 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg"] Jul 14 11:56:23.979006 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:23.978984 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8567a3-54cdd4db66-nzhlg"] Jul 14 11:56:24.678271 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:24.678234 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29b10765-6bea-47d5-a930-50390be8779e" path="/var/lib/kubelet/pods/29b10765-6bea-47d5-a930-50390be8779e/volumes" Jul 14 11:56:49.201868 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.201786 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk"] Jul 14 11:56:49.202683 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.202640 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" podUID="40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" containerName="maas-api" containerID="cri-o://a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c" gracePeriod=30 Jul 14 11:56:49.464574 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.464507 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:56:49.553472 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.553433 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-openshift-service-ca\") pod \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " Jul 14 11:56:49.553643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.553525 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls\") pod \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " Jul 14 11:56:49.553643 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.553568 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnc4f\" (UniqueName: \"kubernetes.io/projected/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-kube-api-access-tnc4f\") pod \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\" (UID: \"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b\") " Jul 14 11:56:49.553872 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.553851 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" (UID: "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:56:49.555764 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.555745 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-kube-api-access-tnc4f" (OuterVolumeSpecName: "kube-api-access-tnc4f") pod "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" (UID: "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b"). InnerVolumeSpecName "kube-api-access-tnc4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:56:49.555889 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.555866 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" (UID: "40f7cf8b-0bfd-4e52-9953-9c02ab869f9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:56:49.654244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.654209 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:56:49.654244 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.654241 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:56:49.654479 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:49.654281 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnc4f\" (UniqueName: \"kubernetes.io/projected/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b-kube-api-access-tnc4f\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:56:50.087336 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.087294 2568 generic.go:356] "Generic (PLEG): container finished" podID="40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" containerID="a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c" exitCode=0 Jul 14 11:56:50.087539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.087345 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" Jul 14 11:56:50.087539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.087375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" event={"ID":"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b","Type":"ContainerDied","Data":"a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c"} Jul 14 11:56:50.087539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.087442 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk" event={"ID":"40f7cf8b-0bfd-4e52-9953-9c02ab869f9b","Type":"ContainerDied","Data":"cfd8a1cf9c328bdc96557f1e30663f8e1d6d8ab86f08b9a24c4d14daac3d6883"} Jul 14 11:56:50.087539 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.087472 2568 scope.go:117] "RemoveContainer" containerID="a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c" Jul 14 11:56:50.097069 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.097040 2568 scope.go:117] "RemoveContainer" containerID="a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c" Jul 14 11:56:50.097361 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:56:50.097340 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c\": container with ID starting with a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c not found: ID does not exist" containerID="a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c" Jul 14 11:56:50.097471 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.097374 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c"} err="failed to get container status \"a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c\": rpc error: code = NotFound desc = could not find container \"a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c\": container with ID starting with a059650fe3be81d6121a6e57606b1f5cf7ed9670813da12d4d5b073bfa26669c not found: ID does not exist" Jul 14 11:56:50.110419 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.110394 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk"] Jul 14 11:56:50.114619 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.114596 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-738ce7-5b6848df86-sd7nk"] Jul 14 11:56:50.679125 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:56:50.679092 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" path="/var/lib/kubelet/pods/40f7cf8b-0bfd-4e52-9953-9c02ab869f9b/volumes" Jul 14 11:57:08.411008 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.410973 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6"] Jul 14 11:57:08.411439 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.411423 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" containerName="maas-api" Jul 14 11:57:08.411439 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.411440 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" containerName="maas-api" Jul 14 11:57:08.411526 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.411463 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b10765-6bea-47d5-a930-50390be8779e" containerName="maas-api" Jul 14 11:57:08.411526 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.411469 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b10765-6bea-47d5-a930-50390be8779e" containerName="maas-api" Jul 14 11:57:08.411602 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.411537 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b10765-6bea-47d5-a930-50390be8779e" containerName="maas-api" Jul 14 11:57:08.411602 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.411547 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="40f7cf8b-0bfd-4e52-9953-9c02ab869f9b" containerName="maas-api" Jul 14 11:57:08.414657 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.414640 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.418146 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.418121 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:57:08.418417 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.418395 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-565f5c-serving-cert\"" Jul 14 11:57:08.427222 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.427199 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6"] Jul 14 11:57:08.525583 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.525548 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf43871-7bd2-45b2-877e-24c57b5804c0-maas-api-tls\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.525732 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.525595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf43871-7bd2-45b2-877e-24c57b5804c0-openshift-service-ca\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.525732 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.525620 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzhz\" (UniqueName: \"kubernetes.io/projected/cdf43871-7bd2-45b2-877e-24c57b5804c0-kube-api-access-btzhz\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.626986 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.626944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf43871-7bd2-45b2-877e-24c57b5804c0-maas-api-tls\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.627177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.627005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf43871-7bd2-45b2-877e-24c57b5804c0-openshift-service-ca\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.627177 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.627045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzhz\" (UniqueName: \"kubernetes.io/projected/cdf43871-7bd2-45b2-877e-24c57b5804c0-kube-api-access-btzhz\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.627753 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.627729 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf43871-7bd2-45b2-877e-24c57b5804c0-openshift-service-ca\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.629550 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.629528 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf43871-7bd2-45b2-877e-24c57b5804c0-maas-api-tls\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.636596 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.636574 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzhz\" (UniqueName: \"kubernetes.io/projected/cdf43871-7bd2-45b2-877e-24c57b5804c0-kube-api-access-btzhz\") pod \"maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.726786 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.726705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:08.873841 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.873815 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6"] Jul 14 11:57:08.875677 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:57:08.875651 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf43871_7bd2_45b2_877e_24c57b5804c0.slice/crio-d4a1ee5d45d69c6ca9b15e51875d96508cee215be9713c2ad9c136f2c85e3c09 WatchSource:0}: Error finding container d4a1ee5d45d69c6ca9b15e51875d96508cee215be9713c2ad9c136f2c85e3c09: Status 404 returned error can't find the container with id d4a1ee5d45d69c6ca9b15e51875d96508cee215be9713c2ad9c136f2c85e3c09 Jul 14 11:57:08.877052 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:08.877034 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:57:09.187932 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:09.187891 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" event={"ID":"cdf43871-7bd2-45b2-877e-24c57b5804c0","Type":"ContainerStarted","Data":"d4a1ee5d45d69c6ca9b15e51875d96508cee215be9713c2ad9c136f2c85e3c09"} Jul 14 11:57:10.193716 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:10.193674 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" event={"ID":"cdf43871-7bd2-45b2-877e-24c57b5804c0","Type":"ContainerStarted","Data":"7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4"} Jul 14 11:57:10.194179 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:10.193733 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:10.215014 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:10.214967 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" podStartSLOduration=1.697948891 podStartE2EDuration="2.214939156s" totalImagesPullingTime="516.990265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:57:10.213676489 +0000 UTC m=+2360.069510666" watchObservedRunningTime="2026-07-14 11:57:10.214939156 +0000 UTC m=+2360.070773383" Jul 14 11:57:16.205029 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.205000 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:16.660869 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.660832 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525"] Jul 14 11:57:16.665164 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.665138 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.668133 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.668110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-290745-serving-cert\"" Jul 14 11:57:16.682487 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.682460 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525"] Jul 14 11:57:16.701829 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.701791 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-openshift-service-ca\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.701829 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.701827 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.701999 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.701846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrrgp\" (UniqueName: \"kubernetes.io/projected/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-kube-api-access-xrrgp\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.802459 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.802429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-openshift-service-ca\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.802644 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.802466 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.802644 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.802486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrrgp\" (UniqueName: \"kubernetes.io/projected/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-kube-api-access-xrrgp\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.802644 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:57:16.802608 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-290745-serving-cert: secret "maas-api-e2e-api-b-290745-serving-cert" not found Jul 14 11:57:16.802769 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:57:16.802713 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls podName:fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2 nodeName:}" failed. No retries permitted until 2026-07-14 11:57:17.302698025 +0000 UTC m=+2367.158532175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls") pod "maas-api-e2e-api-b-290745-75676558d7-lm525" (UID: "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2") : secret "maas-api-e2e-api-b-290745-serving-cert" not found Jul 14 11:57:16.803195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.803173 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-openshift-service-ca\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:16.813859 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:16.813831 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrrgp\" (UniqueName: \"kubernetes.io/projected/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-kube-api-access-xrrgp\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:17.306251 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:17.306217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:17.308984 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:17.308948 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls\") pod \"maas-api-e2e-api-b-290745-75676558d7-lm525\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:17.578998 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:17.578911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:17.726561 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:17.726535 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525"] Jul 14 11:57:17.729273 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:57:17.729233 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa37a4b8_0a25_4b04_bf86_123ebb8e1ca2.slice/crio-2039c019cd098a3d5259827bee4efd1c8ef86d8cf623231b9c4172865ffc6ce8 WatchSource:0}: Error finding container 2039c019cd098a3d5259827bee4efd1c8ef86d8cf623231b9c4172865ffc6ce8: Status 404 returned error can't find the container with id 2039c019cd098a3d5259827bee4efd1c8ef86d8cf623231b9c4172865ffc6ce8 Jul 14 11:57:18.238909 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:18.238875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" event={"ID":"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2","Type":"ContainerStarted","Data":"7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929"} Jul 14 11:57:18.239061 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:18.238924 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" event={"ID":"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2","Type":"ContainerStarted","Data":"2039c019cd098a3d5259827bee4efd1c8ef86d8cf623231b9c4172865ffc6ce8"} Jul 14 11:57:18.239061 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:18.239009 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:18.258323 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:18.258249 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" podStartSLOduration=1.8505702849999999 podStartE2EDuration="2.258234533s" totalImagesPullingTime="407.664248ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:57:18.257176661 +0000 UTC m=+2368.113010834" watchObservedRunningTime="2026-07-14 11:57:18.258234533 +0000 UTC m=+2368.114068696" Jul 14 11:57:24.249058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:24.249026 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:57:38.914325 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:38.914288 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6"] Jul 14 11:57:38.914704 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:38.914549 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" podUID="cdf43871-7bd2-45b2-877e-24c57b5804c0" containerName="maas-api" containerID="cri-o://7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4" gracePeriod=30 Jul 14 11:57:39.172740 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.172681 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:39.300908 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.300877 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btzhz\" (UniqueName: \"kubernetes.io/projected/cdf43871-7bd2-45b2-877e-24c57b5804c0-kube-api-access-btzhz\") pod \"cdf43871-7bd2-45b2-877e-24c57b5804c0\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " Jul 14 11:57:39.301077 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.300932 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf43871-7bd2-45b2-877e-24c57b5804c0-openshift-service-ca\") pod \"cdf43871-7bd2-45b2-877e-24c57b5804c0\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " Jul 14 11:57:39.301077 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.300991 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf43871-7bd2-45b2-877e-24c57b5804c0-maas-api-tls\") pod \"cdf43871-7bd2-45b2-877e-24c57b5804c0\" (UID: \"cdf43871-7bd2-45b2-877e-24c57b5804c0\") " Jul 14 11:57:39.301398 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.301371 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf43871-7bd2-45b2-877e-24c57b5804c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdf43871-7bd2-45b2-877e-24c57b5804c0" (UID: "cdf43871-7bd2-45b2-877e-24c57b5804c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:57:39.303145 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.303122 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf43871-7bd2-45b2-877e-24c57b5804c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdf43871-7bd2-45b2-877e-24c57b5804c0" (UID: "cdf43871-7bd2-45b2-877e-24c57b5804c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:57:39.303216 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.303160 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf43871-7bd2-45b2-877e-24c57b5804c0-kube-api-access-btzhz" (OuterVolumeSpecName: "kube-api-access-btzhz") pod "cdf43871-7bd2-45b2-877e-24c57b5804c0" (UID: "cdf43871-7bd2-45b2-877e-24c57b5804c0"). InnerVolumeSpecName "kube-api-access-btzhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:57:39.343623 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.343593 2568 generic.go:356] "Generic (PLEG): container finished" podID="cdf43871-7bd2-45b2-877e-24c57b5804c0" containerID="7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4" exitCode=0 Jul 14 11:57:39.343794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.343645 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" Jul 14 11:57:39.343794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.343653 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" event={"ID":"cdf43871-7bd2-45b2-877e-24c57b5804c0","Type":"ContainerDied","Data":"7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4"} Jul 14 11:57:39.343794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.343696 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6" event={"ID":"cdf43871-7bd2-45b2-877e-24c57b5804c0","Type":"ContainerDied","Data":"d4a1ee5d45d69c6ca9b15e51875d96508cee215be9713c2ad9c136f2c85e3c09"} Jul 14 11:57:39.343794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.343716 2568 scope.go:117] "RemoveContainer" containerID="7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4" Jul 14 11:57:39.353159 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.353141 2568 scope.go:117] "RemoveContainer" containerID="7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4" Jul 14 11:57:39.353426 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:57:39.353409 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4\": container with ID starting with 7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4 not found: ID does not exist" containerID="7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4" Jul 14 11:57:39.353484 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.353433 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4"} err="failed to get container status \"7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4\": rpc error: code = NotFound desc = could not find container \"7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4\": container with ID starting with 7550bfb8015ffd6ccfe5eda717058bc8b5c16cd125b3195c0288f0ed9960aba4 not found: ID does not exist" Jul 14 11:57:39.367448 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.367424 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6"] Jul 14 11:57:39.371859 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.371840 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-565f5c-597c9485ff-2g9l6"] Jul 14 11:57:39.402598 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.402574 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdf43871-7bd2-45b2-877e-24c57b5804c0-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:57:39.402598 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.402599 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdf43871-7bd2-45b2-877e-24c57b5804c0-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:57:39.402742 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:39.402608 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btzhz\" (UniqueName: \"kubernetes.io/projected/cdf43871-7bd2-45b2-877e-24c57b5804c0-kube-api-access-btzhz\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:57:40.678114 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:40.678071 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf43871-7bd2-45b2-877e-24c57b5804c0" path="/var/lib/kubelet/pods/cdf43871-7bd2-45b2-877e-24c57b5804c0/volumes" Jul 14 11:57:50.780526 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:50.780406 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:57:50.793588 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:50.786278 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 11:57:50.793588 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:50.787689 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:57:50.793709 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:57:50.793619 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 11:58:05.473996 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.473965 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525"] Jul 14 11:58:05.474615 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.474278 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" podUID="fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" containerName="maas-api" containerID="cri-o://7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929" gracePeriod=30 Jul 14 11:58:05.741302 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.741276 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:58:05.826425 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.826381 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-openshift-service-ca\") pod \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " Jul 14 11:58:05.826610 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.826451 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrrgp\" (UniqueName: \"kubernetes.io/projected/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-kube-api-access-xrrgp\") pod \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " Jul 14 11:58:05.826610 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.826504 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls\") pod \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\" (UID: \"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2\") " Jul 14 11:58:05.826777 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.826748 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" (UID: "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:58:05.826867 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.826849 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:58:05.828759 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.828730 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-kube-api-access-xrrgp" (OuterVolumeSpecName: "kube-api-access-xrrgp") pod "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" (UID: "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2"). InnerVolumeSpecName "kube-api-access-xrrgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:58:05.828759 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.828738 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" (UID: "fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:58:05.927707 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.927679 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrrgp\" (UniqueName: \"kubernetes.io/projected/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-kube-api-access-xrrgp\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:58:05.927707 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:05.927706 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 11:58:06.486940 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.486901 2568 generic.go:356] "Generic (PLEG): container finished" podID="fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" containerID="7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929" exitCode=0 Jul 14 11:58:06.487468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.486964 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" Jul 14 11:58:06.487468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.486988 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" event={"ID":"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2","Type":"ContainerDied","Data":"7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929"} Jul 14 11:58:06.487468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.487095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525" event={"ID":"fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2","Type":"ContainerDied","Data":"2039c019cd098a3d5259827bee4efd1c8ef86d8cf623231b9c4172865ffc6ce8"} Jul 14 11:58:06.487468 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.487129 2568 scope.go:117] "RemoveContainer" containerID="7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929" Jul 14 11:58:06.496859 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.496840 2568 scope.go:117] "RemoveContainer" containerID="7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929" Jul 14 11:58:06.497140 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:58:06.497118 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929\": container with ID starting with 7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929 not found: ID does not exist" containerID="7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929" Jul 14 11:58:06.497230 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.497145 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929"} err="failed to get container status \"7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929\": rpc error: code = NotFound desc = could not find container \"7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929\": container with ID starting with 7da913314982836e45f996d2f1f293718cdd906a0b574a2782ea0ddbf4b52929 not found: ID does not exist" Jul 14 11:58:06.509357 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.509291 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525"] Jul 14 11:58:06.513134 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.513109 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-290745-75676558d7-lm525"] Jul 14 11:58:06.678629 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:06.678600 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" path="/var/lib/kubelet/pods/fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2/volumes" Jul 14 11:58:24.899247 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.899164 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5"] Jul 14 11:58:24.899876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.899853 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" containerName="maas-api" Jul 14 11:58:24.899876 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.899877 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" containerName="maas-api" Jul 14 11:58:24.900057 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.899905 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf43871-7bd2-45b2-877e-24c57b5804c0" containerName="maas-api" Jul 14 11:58:24.900057 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.899913 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf43871-7bd2-45b2-877e-24c57b5804c0" containerName="maas-api" Jul 14 11:58:24.900168 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.900061 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf43871-7bd2-45b2-877e-24c57b5804c0" containerName="maas-api" Jul 14 11:58:24.900168 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.900076 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa37a4b8-0a25-4b04-bf86-123ebb8e1ca2" containerName="maas-api" Jul 14 11:58:24.904665 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.904642 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:24.908270 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.908230 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-1cc796-serving-cert\"" Jul 14 11:58:24.908659 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.908641 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:58:24.917971 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.917949 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5"] Jul 14 11:58:24.986022 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.985984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:24.986195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.986051 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:24.986195 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:24.986099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgm2d\" (UniqueName: \"kubernetes.io/projected/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-kube-api-access-qgm2d\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.086565 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.086528 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgm2d\" (UniqueName: \"kubernetes.io/projected/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-kube-api-access-qgm2d\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.086745 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.086601 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.086745 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.086648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.087440 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.087383 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.089192 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.089170 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.099405 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.099378 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgm2d\" (UniqueName: \"kubernetes.io/projected/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-kube-api-access-qgm2d\") pod \"maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.217607 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.217524 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:25.359987 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.359958 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5"] Jul 14 11:58:25.361944 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:58:25.361916 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4d7956_b4b1_4678_94f4_93da9fe9d8dd.slice/crio-870fa0fbeb7ec653d9c2113ef7a6beeff5bba6090d482d92f11224ec0ec0e56c WatchSource:0}: Error finding container 870fa0fbeb7ec653d9c2113ef7a6beeff5bba6090d482d92f11224ec0ec0e56c: Status 404 returned error can't find the container with id 870fa0fbeb7ec653d9c2113ef7a6beeff5bba6090d482d92f11224ec0ec0e56c Jul 14 11:58:25.585268 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:25.585218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" event={"ID":"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd","Type":"ContainerStarted","Data":"870fa0fbeb7ec653d9c2113ef7a6beeff5bba6090d482d92f11224ec0ec0e56c"} Jul 14 11:58:26.591690 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:26.591649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" event={"ID":"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd","Type":"ContainerStarted","Data":"ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29"} Jul 14 11:58:26.592169 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:26.591757 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:32.603105 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.603069 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 11:58:32.623280 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.623197 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" podStartSLOduration=8.24436775 podStartE2EDuration="8.623177648s" totalImagesPullingTime="378.809898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:58:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:58:26.633459743 +0000 UTC m=+2436.489293914" watchObservedRunningTime="2026-07-14 11:58:32.623177648 +0000 UTC m=+2442.479011821" Jul 14 11:58:32.909510 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.909415 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp"] Jul 14 11:58:32.913625 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.913598 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:32.916678 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.916651 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0508a3-serving-cert\"" Jul 14 11:58:32.925701 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.925676 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp"] Jul 14 11:58:32.960860 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.960832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k28pz\" (UniqueName: \"kubernetes.io/projected/726506f2-d193-4b86-909b-f052b9e983a6-kube-api-access-k28pz\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:32.961010 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.960902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:32.961010 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:32.960964 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726506f2-d193-4b86-909b-f052b9e983a6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.061794 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.061752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726506f2-d193-4b86-909b-f052b9e983a6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.061985 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.061847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k28pz\" (UniqueName: \"kubernetes.io/projected/726506f2-d193-4b86-909b-f052b9e983a6-kube-api-access-k28pz\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.061985 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.061906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.062107 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:58:33.062020 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-serving-cert: secret "maas-api-e2e-shared-b-0508a3-serving-cert" not found Jul 14 11:58:33.062107 ip-10-0-136-255 kubenswrapper[2568]: E0714 11:58:33.062089 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls podName:726506f2-d193-4b86-909b-f052b9e983a6 nodeName:}" failed. No retries permitted until 2026-07-14 11:58:33.562068482 +0000 UTC m=+2443.417902631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls") pod "maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" (UID: "726506f2-d193-4b86-909b-f052b9e983a6") : secret "maas-api-e2e-shared-b-0508a3-serving-cert" not found Jul 14 11:58:33.062389 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.062368 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726506f2-d193-4b86-909b-f052b9e983a6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.079027 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.078994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k28pz\" (UniqueName: \"kubernetes.io/projected/726506f2-d193-4b86-909b-f052b9e983a6-kube-api-access-k28pz\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.565624 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.565574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.568220 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.568194 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls\") pod \"maas-api-e2e-shared-b-0508a3-787469f576-nv9gp\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.828952 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.828858 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:33.973369 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:33.973345 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp"] Jul 14 11:58:33.975346 ip-10-0-136-255 kubenswrapper[2568]: W0714 11:58:33.975304 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726506f2_d193_4b86_909b_f052b9e983a6.slice/crio-431fb66f5b9b9baa162fd9fa2066cdb441a7637a63d5cd3f536b14cdda643ff2 WatchSource:0}: Error finding container 431fb66f5b9b9baa162fd9fa2066cdb441a7637a63d5cd3f536b14cdda643ff2: Status 404 returned error can't find the container with id 431fb66f5b9b9baa162fd9fa2066cdb441a7637a63d5cd3f536b14cdda643ff2 Jul 14 11:58:34.637383 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:34.637344 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" event={"ID":"726506f2-d193-4b86-909b-f052b9e983a6","Type":"ContainerStarted","Data":"63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0"} Jul 14 11:58:34.637547 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:34.637400 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" event={"ID":"726506f2-d193-4b86-909b-f052b9e983a6","Type":"ContainerStarted","Data":"431fb66f5b9b9baa162fd9fa2066cdb441a7637a63d5cd3f536b14cdda643ff2"} Jul 14 11:58:34.637547 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:34.637451 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 11:58:34.669491 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:34.669392 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" podStartSLOduration=2.243582101 podStartE2EDuration="2.669377139s" totalImagesPullingTime="425.795038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:58:34.668029121 +0000 UTC m=+2444.523863295" watchObservedRunningTime="2026-07-14 11:58:34.669377139 +0000 UTC m=+2444.525211322" Jul 14 11:58:40.648058 ip-10-0-136-255 kubenswrapper[2568]: I0714 11:58:40.648031 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 12:00:00.139028 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.138994 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5"] Jul 14 12:00:00.142508 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.142483 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" Jul 14 12:00:00.144927 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.144905 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6"] Jul 14 12:00:00.145779 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.145764 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\"" Jul 14 12:00:00.148331 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.148312 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" Jul 14 12:00:00.150933 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.150908 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5"] Jul 14 12:00:00.156583 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.156556 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6"] Jul 14 12:00:00.157498 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.157476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" Jul 14 12:00:00.165813 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.165796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" Jul 14 12:00:00.312804 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.312774 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5"] Jul 14 12:00:00.316167 ip-10-0-136-255 kubenswrapper[2568]: W0714 12:00:00.316107 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf313475a_bfe5_4030_847b_35177a268b9d.slice/crio-e2ea4c4c82e63871c4745bd8820a2fda3064ef2c77e488841cab3c516d91ddfe WatchSource:0}: Error finding container e2ea4c4c82e63871c4745bd8820a2fda3064ef2c77e488841cab3c516d91ddfe: Status 404 returned error can't find the container with id e2ea4c4c82e63871c4745bd8820a2fda3064ef2c77e488841cab3c516d91ddfe Jul 14 12:00:00.344691 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:00.344664 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6"] Jul 14 12:00:00.353424 ip-10-0-136-255 kubenswrapper[2568]: W0714 12:00:00.353400 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0fe0015_131c_4335_b647_b4a826f27501.slice/crio-e79c90001bd5e7e2e49a418c947c9be267d3627a80a60ac7eca66fa31c232edf WatchSource:0}: Error finding container e79c90001bd5e7e2e49a418c947c9be267d3627a80a60ac7eca66fa31c232edf: Status 404 returned error can't find the container with id e79c90001bd5e7e2e49a418c947c9be267d3627a80a60ac7eca66fa31c232edf Jul 14 12:00:01.085276 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:01.085224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" event={"ID":"e0fe0015-131c-4335-b647-b4a826f27501","Type":"ContainerStarted","Data":"a420be363b698ca1f1d0ae2e94a092e4bea949617da5be2472d304ded5ad555c"} Jul 14 12:00:01.085470 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:01.085309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" event={"ID":"e0fe0015-131c-4335-b647-b4a826f27501","Type":"ContainerStarted","Data":"e79c90001bd5e7e2e49a418c947c9be267d3627a80a60ac7eca66fa31c232edf"} Jul 14 12:00:01.086798 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:01.086777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" event={"ID":"f313475a-bfe5-4030-847b-35177a268b9d","Type":"ContainerStarted","Data":"2e174fd1c5def8ddb185385567f7c0cee2ad425bb73dc38b44a82f40737321d9"} Jul 14 12:00:01.086898 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:01.086811 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" event={"ID":"f313475a-bfe5-4030-847b-35177a268b9d","Type":"ContainerStarted","Data":"e2ea4c4c82e63871c4745bd8820a2fda3064ef2c77e488841cab3c516d91ddfe"} Jul 14 12:00:01.105166 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:01.105113 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" podStartSLOduration=1.105098613 podStartE2EDuration="1.105098613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:00:01.103544831 +0000 UTC m=+2530.959379005" watchObservedRunningTime="2026-07-14 12:00:01.105098613 +0000 UTC m=+2530.960932784" Jul 14 12:00:01.119799 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:01.119678 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" podStartSLOduration=1.119662591 podStartE2EDuration="1.119662591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:00:01.119587804 +0000 UTC m=+2530.975421976" watchObservedRunningTime="2026-07-14 12:00:01.119662591 +0000 UTC m=+2530.975496763" Jul 14 12:00:02.092957 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:02.092925 2568 generic.go:356] "Generic (PLEG): container finished" podID="e0fe0015-131c-4335-b647-b4a826f27501" containerID="a420be363b698ca1f1d0ae2e94a092e4bea949617da5be2472d304ded5ad555c" exitCode=0 Jul 14 12:00:02.093442 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:02.093011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" event={"ID":"e0fe0015-131c-4335-b647-b4a826f27501","Type":"ContainerDied","Data":"a420be363b698ca1f1d0ae2e94a092e4bea949617da5be2472d304ded5ad555c"} Jul 14 12:00:02.094472 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:02.094450 2568 generic.go:356] "Generic (PLEG): container finished" podID="f313475a-bfe5-4030-847b-35177a268b9d" containerID="2e174fd1c5def8ddb185385567f7c0cee2ad425bb73dc38b44a82f40737321d9" exitCode=0 Jul 14 12:00:02.094557 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:02.094485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" event={"ID":"f313475a-bfe5-4030-847b-35177a268b9d","Type":"ContainerDied","Data":"2e174fd1c5def8ddb185385567f7c0cee2ad425bb73dc38b44a82f40737321d9"} Jul 14 12:00:03.259705 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:03.259678 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" Jul 14 12:00:03.262682 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:03.262645 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" Jul 14 12:00:04.104987 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:04.104959 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" Jul 14 12:00:04.105180 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:04.104999 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733840-2j9g5" event={"ID":"f313475a-bfe5-4030-847b-35177a268b9d","Type":"ContainerDied","Data":"e2ea4c4c82e63871c4745bd8820a2fda3064ef2c77e488841cab3c516d91ddfe"} Jul 14 12:00:04.105180 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:04.105045 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2ea4c4c82e63871c4745bd8820a2fda3064ef2c77e488841cab3c516d91ddfe" Jul 14 12:00:04.106289 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:04.106250 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" event={"ID":"e0fe0015-131c-4335-b647-b4a826f27501","Type":"ContainerDied","Data":"e79c90001bd5e7e2e49a418c947c9be267d3627a80a60ac7eca66fa31c232edf"} Jul 14 12:00:04.106383 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:04.106304 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e79c90001bd5e7e2e49a418c947c9be267d3627a80a60ac7eca66fa31c232edf" Jul 14 12:00:04.106383 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:00:04.106275 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6" Jul 14 12:02:50.811859 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:02:50.811831 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 12:02:50.819772 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:02:50.819745 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 12:02:50.820166 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:02:50.820146 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 12:02:50.827076 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:02:50.827059 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ngnhm_45c57b79-de41-4891-a513-7102b9c8ffc3/ovn-acl-logging/0.log" Jul 14 12:04:25.150947 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.150863 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7264-140675705691312"] Jul 14 12:04:25.151367 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.151321 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f313475a-bfe5-4030-847b-35177a268b9d" containerName="cleanup" Jul 14 12:04:25.151367 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.151334 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f313475a-bfe5-4030-847b-35177a268b9d" containerName="cleanup" Jul 14 12:04:25.151367 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.151346 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0fe0015-131c-4335-b647-b4a826f27501" containerName="cleanup" Jul 14 12:04:25.151367 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.151352 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0fe0015-131c-4335-b647-b4a826f27501" containerName="cleanup" Jul 14 12:04:25.151505 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.151428 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0fe0015-131c-4335-b647-b4a826f27501" containerName="cleanup" Jul 14 12:04:25.151505 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.151437 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f313475a-bfe5-4030-847b-35177a268b9d" containerName="cleanup" Jul 14 12:04:25.154673 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.154655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:25.158005 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.157985 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-dn6f4\"" Jul 14 12:04:25.161972 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.161950 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705691312"] Jul 14 12:04:25.173651 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.173624 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95nlf\" (UniqueName: \"kubernetes.io/projected/0760a53a-55f8-4f97-8853-061e563d16b5-kube-api-access-95nlf\") pod \"test-curl-7264-140675705691312\" (UID: \"0760a53a-55f8-4f97-8853-061e563d16b5\") " pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:25.274508 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.274468 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95nlf\" (UniqueName: \"kubernetes.io/projected/0760a53a-55f8-4f97-8853-061e563d16b5-kube-api-access-95nlf\") pod \"test-curl-7264-140675705691312\" (UID: \"0760a53a-55f8-4f97-8853-061e563d16b5\") " pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:25.283742 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.283711 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95nlf\" (UniqueName: \"kubernetes.io/projected/0760a53a-55f8-4f97-8853-061e563d16b5-kube-api-access-95nlf\") pod \"test-curl-7264-140675705691312\" (UID: \"0760a53a-55f8-4f97-8853-061e563d16b5\") " pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:25.464562 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.464471 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:25.602925 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.602808 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705691312"] Jul 14 12:04:25.605360 ip-10-0-136-255 kubenswrapper[2568]: W0714 12:04:25.605319 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0760a53a_55f8_4f97_8853_061e563d16b5.slice/crio-d74ac9a1256492ac3e232599372e8d5a8b50196cd58fd5c9a2f3bcb87b082069 WatchSource:0}: Error finding container d74ac9a1256492ac3e232599372e8d5a8b50196cd58fd5c9a2f3bcb87b082069: Status 404 returned error can't find the container with id d74ac9a1256492ac3e232599372e8d5a8b50196cd58fd5c9a2f3bcb87b082069 Jul 14 12:04:25.606735 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:25.606715 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 12:04:26.460719 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:26.460688 2568 generic.go:356] "Generic (PLEG): container finished" podID="0760a53a-55f8-4f97-8853-061e563d16b5" containerID="b0ea0f6c219141c7288d942d28ceb11183e0bd5af808ce45a92870659a3efeb6" exitCode=0 Jul 14 12:04:26.461123 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:26.460727 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705691312" event={"ID":"0760a53a-55f8-4f97-8853-061e563d16b5","Type":"ContainerDied","Data":"b0ea0f6c219141c7288d942d28ceb11183e0bd5af808ce45a92870659a3efeb6"} Jul 14 12:04:26.461123 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:26.460753 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705691312" event={"ID":"0760a53a-55f8-4f97-8853-061e563d16b5","Type":"ContainerStarted","Data":"d74ac9a1256492ac3e232599372e8d5a8b50196cd58fd5c9a2f3bcb87b082069"} Jul 14 12:04:27.611124 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.611096 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:27.645738 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.645710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7264-140675705691312_0760a53a-55f8-4f97-8853-061e563d16b5/test-curl-7264-140675705691312/0.log" Jul 14 12:04:27.670343 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.670311 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7264-140675705691312"] Jul 14 12:04:27.674969 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.674945 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7264-140675705691312"] Jul 14 12:04:27.698116 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.698094 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95nlf\" (UniqueName: \"kubernetes.io/projected/0760a53a-55f8-4f97-8853-061e563d16b5-kube-api-access-95nlf\") pod \"0760a53a-55f8-4f97-8853-061e563d16b5\" (UID: \"0760a53a-55f8-4f97-8853-061e563d16b5\") " Jul 14 12:04:27.700379 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.700351 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0760a53a-55f8-4f97-8853-061e563d16b5-kube-api-access-95nlf" (OuterVolumeSpecName: "kube-api-access-95nlf") pod "0760a53a-55f8-4f97-8853-061e563d16b5" (UID: "0760a53a-55f8-4f97-8853-061e563d16b5"). InnerVolumeSpecName "kube-api-access-95nlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:04:27.799082 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:27.799054 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95nlf\" (UniqueName: \"kubernetes.io/projected/0760a53a-55f8-4f97-8853-061e563d16b5-kube-api-access-95nlf\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:04:28.471752 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:28.471713 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:28.471752 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:28.471732 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d74ac9a1256492ac3e232599372e8d5a8b50196cd58fd5c9a2f3bcb87b082069" Jul 14 12:04:28.481898 ip-10-0-136-255 kubenswrapper[2568]: E0714 12:04:28.481866 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7264-140675705691312\" is forbidden: User \"system:node:ip-10-0-136-255.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-255.ec2.internal' and this object" podUID="0760a53a-55f8-4f97-8853-061e563d16b5" pod="openshift-ingress/test-curl-7264-140675705691312" Jul 14 12:04:28.678293 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:28.678240 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0760a53a-55f8-4f97-8853-061e563d16b5" path="/var/lib/kubelet/pods/0760a53a-55f8-4f97-8853-061e563d16b5/volumes" Jul 14 12:04:53.609999 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.609959 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5"] Jul 14 12:04:53.610500 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.610239 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" podUID="3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" containerName="maas-api" containerID="cri-o://ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29" gracePeriod=30 Jul 14 12:04:53.864442 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.864380 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 12:04:53.930033 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.930001 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-openshift-service-ca\") pod \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " Jul 14 12:04:53.930225 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.930072 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgm2d\" (UniqueName: \"kubernetes.io/projected/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-kube-api-access-qgm2d\") pod \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " Jul 14 12:04:53.930341 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.930313 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-maas-api-tls\") pod \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\" (UID: \"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd\") " Jul 14 12:04:53.930490 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.930464 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" (UID: "3b4d7956-b4b1-4678-94f4-93da9fe9d8dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 12:04:53.930633 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.930618 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:04:53.932344 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.932322 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-kube-api-access-qgm2d" (OuterVolumeSpecName: "kube-api-access-qgm2d") pod "3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" (UID: "3b4d7956-b4b1-4678-94f4-93da9fe9d8dd"). InnerVolumeSpecName "kube-api-access-qgm2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:04:53.932422 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:53.932379 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" (UID: "3b4d7956-b4b1-4678-94f4-93da9fe9d8dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 12:04:54.031613 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.031577 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:04:54.031613 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.031607 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgm2d\" (UniqueName: \"kubernetes.io/projected/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd-kube-api-access-qgm2d\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:04:54.605273 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.605220 2568 generic.go:356] "Generic (PLEG): container finished" podID="3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" containerID="ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29" exitCode=0 Jul 14 12:04:54.605456 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.605295 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" Jul 14 12:04:54.605456 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.605289 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" event={"ID":"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd","Type":"ContainerDied","Data":"ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29"} Jul 14 12:04:54.605456 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.605356 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5" event={"ID":"3b4d7956-b4b1-4678-94f4-93da9fe9d8dd","Type":"ContainerDied","Data":"870fa0fbeb7ec653d9c2113ef7a6beeff5bba6090d482d92f11224ec0ec0e56c"} Jul 14 12:04:54.605456 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.605379 2568 scope.go:117] "RemoveContainer" containerID="ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29" Jul 14 12:04:54.615339 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.615160 2568 scope.go:117] "RemoveContainer" containerID="ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29" Jul 14 12:04:54.615583 ip-10-0-136-255 kubenswrapper[2568]: E0714 12:04:54.615471 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29\": container with ID starting with ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29 not found: ID does not exist" containerID="ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29" Jul 14 12:04:54.615583 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.615498 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29"} err="failed to get container status \"ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29\": rpc error: code = NotFound desc = could not find container \"ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29\": container with ID starting with ee5d53eccd113e9764f48f536094961bfd67a6a07df64decfbc35166855ebd29 not found: ID does not exist" Jul 14 12:04:54.628481 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.628452 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5"] Jul 14 12:04:54.632210 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.632190 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-1cc796-779fdfdcb5-h7vp5"] Jul 14 12:04:54.678785 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:04:54.678760 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" path="/var/lib/kubelet/pods/3b4d7956-b4b1-4678-94f4-93da9fe9d8dd/volumes" Jul 14 12:05:19.819976 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:19.819931 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp"] Jul 14 12:05:19.820589 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:19.820185 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" podUID="726506f2-d193-4b86-909b-f052b9e983a6" containerName="maas-api" containerID="cri-o://63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0" gracePeriod=30 Jul 14 12:05:19.851665 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:19.851641 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6"] Jul 14 12:05:19.855394 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:19.855371 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0508a3-29733840-7wqh6"] Jul 14 12:05:20.075746 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.075689 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 12:05:20.160475 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.160446 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726506f2-d193-4b86-909b-f052b9e983a6-openshift-service-ca\") pod \"726506f2-d193-4b86-909b-f052b9e983a6\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " Jul 14 12:05:20.160671 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.160578 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k28pz\" (UniqueName: \"kubernetes.io/projected/726506f2-d193-4b86-909b-f052b9e983a6-kube-api-access-k28pz\") pod \"726506f2-d193-4b86-909b-f052b9e983a6\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " Jul 14 12:05:20.160671 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.160614 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls\") pod \"726506f2-d193-4b86-909b-f052b9e983a6\" (UID: \"726506f2-d193-4b86-909b-f052b9e983a6\") " Jul 14 12:05:20.160814 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.160792 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/726506f2-d193-4b86-909b-f052b9e983a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "726506f2-d193-4b86-909b-f052b9e983a6" (UID: "726506f2-d193-4b86-909b-f052b9e983a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 12:05:20.162719 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.162689 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "726506f2-d193-4b86-909b-f052b9e983a6" (UID: "726506f2-d193-4b86-909b-f052b9e983a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 12:05:20.162828 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.162784 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726506f2-d193-4b86-909b-f052b9e983a6-kube-api-access-k28pz" (OuterVolumeSpecName: "kube-api-access-k28pz") pod "726506f2-d193-4b86-909b-f052b9e983a6" (UID: "726506f2-d193-4b86-909b-f052b9e983a6"). InnerVolumeSpecName "kube-api-access-k28pz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:05:20.262045 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.262005 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k28pz\" (UniqueName: \"kubernetes.io/projected/726506f2-d193-4b86-909b-f052b9e983a6-kube-api-access-k28pz\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:05:20.262045 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.262038 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/726506f2-d193-4b86-909b-f052b9e983a6-maas-api-tls\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:05:20.262045 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.262051 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/726506f2-d193-4b86-909b-f052b9e983a6-openshift-service-ca\") on node \"ip-10-0-136-255.ec2.internal\" DevicePath \"\"" Jul 14 12:05:20.679384 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.679349 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0fe0015-131c-4335-b647-b4a826f27501" path="/var/lib/kubelet/pods/e0fe0015-131c-4335-b647-b4a826f27501/volumes" Jul 14 12:05:20.735186 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.735147 2568 generic.go:356] "Generic (PLEG): container finished" podID="726506f2-d193-4b86-909b-f052b9e983a6" containerID="63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0" exitCode=0 Jul 14 12:05:20.735342 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.735195 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" Jul 14 12:05:20.735342 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.735239 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" event={"ID":"726506f2-d193-4b86-909b-f052b9e983a6","Type":"ContainerDied","Data":"63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0"} Jul 14 12:05:20.735342 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.735316 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp" event={"ID":"726506f2-d193-4b86-909b-f052b9e983a6","Type":"ContainerDied","Data":"431fb66f5b9b9baa162fd9fa2066cdb441a7637a63d5cd3f536b14cdda643ff2"} Jul 14 12:05:20.735342 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.735341 2568 scope.go:117] "RemoveContainer" containerID="63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0" Jul 14 12:05:20.744156 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.744138 2568 scope.go:117] "RemoveContainer" containerID="63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0" Jul 14 12:05:20.744453 ip-10-0-136-255 kubenswrapper[2568]: E0714 12:05:20.744433 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0\": container with ID starting with 63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0 not found: ID does not exist" containerID="63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0" Jul 14 12:05:20.744508 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.744461 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0"} err="failed to get container status \"63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0\": rpc error: code = NotFound desc = could not find container \"63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0\": container with ID starting with 63c0c19d6c37225e350766b040173343f3d883b669976793922e462b6bda92d0 not found: ID does not exist" Jul 14 12:05:20.753554 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.753531 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp"] Jul 14 12:05:20.761300 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:20.758761 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0508a3-787469f576-nv9gp"] Jul 14 12:05:22.678862 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:22.678829 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="726506f2-d193-4b86-909b-f052b9e983a6" path="/var/lib/kubelet/pods/726506f2-d193-4b86-909b-f052b9e983a6/volumes" Jul 14 12:05:40.460364 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:40.460333 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vp8rc_76879cef-26ef-4c81-adef-9fa7a7fa7a43/manager/0.log" Jul 14 12:05:40.593689 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:40.593658 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-q5dc9_94c6e106-e6c7-47ea-8682-f1c54fc2d665/manager/0.log" Jul 14 12:05:40.835347 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:40.835323 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fsrww_278a93c6-7346-469e-a8e2-b5f7abc77f11/server/0.log" Jul 14 12:05:40.972414 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:40.972381 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d2jfx_08aba400-78d1-44be-8921-7112772a70aa/manager/0.log" Jul 14 12:05:41.345512 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:41.345485 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-zwpqd_e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1/copy-manifests/0.log" Jul 14 12:05:41.366390 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:41.366359 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-zwpqd_e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1/manager/0.log" Jul 14 12:05:41.859338 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:41.859312 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733810-lbh2m_cb408680-76f3-432e-b2e7-a2c1a47c40ca/cleanup/0.log" Jul 14 12:05:42.079096 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:42.079068 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733840-2j9g5_f313475a-bfe5-4030-847b-35177a268b9d/cleanup/0.log" Jul 14 12:05:42.302985 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:42.302960 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-e8442e03-ms2vn_cf326e7c-575a-4bc8-939c-0f139b290745/revoke-keys/0.log" Jul 14 12:05:43.606180 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:43.606147 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-nvzt5_ebf43b49-05a0-4536-adbe-87018330f062/postgres/0.log" Jul 14 12:05:45.059268 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:45.059235 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qwt96_83575570-bb14-4d4a-ace7-8665aed885e1/manager/0.log" Jul 14 12:05:45.427915 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:45.427830 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2d2x4_ba4c0cca-026e-49dc-8530-83ddc5b0addf/registry-server/0.log" Jul 14 12:05:45.561859 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:45.561817 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wqkc5_8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c/manager/0.log" Jul 14 12:05:45.810942 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:45.810902 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-br4xs_598aa189-a6ed-4c6b-9dc2-289060fe9b74/manager/0.log" Jul 14 12:05:46.451343 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:46.451308 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-9b5zc_fa33eac3-211c-4b2f-a40b-04564300efd9/kube-auth-proxy/0.log" Jul 14 12:05:46.824685 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:46.824652 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qprhx_bb482966-a541-4ffb-a8d9-3745dad2fde3/payload-pre-processing/0.log" Jul 14 12:05:46.948375 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:46.948340 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-rpmgm_ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c/payload-processing/0.log" Jul 14 12:05:47.062465 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:47.062437 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794fd9f96d-dt6lg_00628d94-4a03-456d-8c41-0ab813236ad1/router/0.log" Jul 14 12:05:47.400843 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:47.400804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-f6mqs_0a1973b4-28dc-4b26-bf32-e00f8b5f1439/main/0.log" Jul 14 12:05:47.841187 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:47.841151 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-c2fz4_ebb222cb-761a-4ff2-ab0a-9b7c98694780/main/0.log" Jul 14 12:05:54.863879 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:54.863845 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2qzv4_7e1eb7e2-1e05-4af6-93f1-894e406169c9/global-pull-secret-syncer/0.log" Jul 14 12:05:55.052979 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:55.052945 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8rlv7_296e9775-ef62-455d-9513-1fd1e4358764/konnectivity-agent/0.log" Jul 14 12:05:55.143316 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:05:55.143222 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-255.ec2.internal_525ade16321a29f4f96b69f63298cd87/haproxy/0.log" Jul 14 12:06:00.256547 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:00.256517 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qwt96_83575570-bb14-4d4a-ace7-8665aed885e1/manager/0.log" Jul 14 12:06:00.383066 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:00.383020 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2d2x4_ba4c0cca-026e-49dc-8530-83ddc5b0addf/registry-server/0.log" Jul 14 12:06:00.527268 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:00.527172 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wqkc5_8235bfdc-ea6c-4f3d-9c3f-bf98f12da26c/manager/0.log" Jul 14 12:06:00.653303 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:00.653248 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-br4xs_598aa189-a6ed-4c6b-9dc2-289060fe9b74/manager/0.log" Jul 14 12:06:02.590748 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:02.590713 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-rjc8c_17070e67-dd32-4d46-a9c1-7e2d634e2ccd/cluster-monitoring-operator/0.log" Jul 14 12:06:02.769725 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:02.769691 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k7r8r_4464b705-8365-4a4f-ba5b-935b8c09e6ba/node-exporter/0.log" Jul 14 12:06:02.790514 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:02.790480 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k7r8r_4464b705-8365-4a4f-ba5b-935b8c09e6ba/kube-rbac-proxy/0.log" Jul 14 12:06:02.812993 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:02.812973 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k7r8r_4464b705-8365-4a4f-ba5b-935b8c09e6ba/init-textfile/0.log" Jul 14 12:06:05.040265 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040223 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4"] Jul 14 12:06:05.040637 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040615 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" containerName="maas-api" Jul 14 12:06:05.040637 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040627 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" containerName="maas-api" Jul 14 12:06:05.040637 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040638 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726506f2-d193-4b86-909b-f052b9e983a6" containerName="maas-api" Jul 14 12:06:05.040753 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040643 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="726506f2-d193-4b86-909b-f052b9e983a6" containerName="maas-api" Jul 14 12:06:05.040753 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040651 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0760a53a-55f8-4f97-8853-061e563d16b5" containerName="test-curl-7264-140675705691312" Jul 14 12:06:05.040753 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040656 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0760a53a-55f8-4f97-8853-061e563d16b5" containerName="test-curl-7264-140675705691312" Jul 14 12:06:05.040753 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040731 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4d7956-b4b1-4678-94f4-93da9fe9d8dd" containerName="maas-api" Jul 14 12:06:05.040753 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040743 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="726506f2-d193-4b86-909b-f052b9e983a6" containerName="maas-api" Jul 14 12:06:05.040753 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.040754 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0760a53a-55f8-4f97-8853-061e563d16b5" containerName="test-curl-7264-140675705691312" Jul 14 12:06:05.043872 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.043853 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.047274 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.047231 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5q25\"/\"default-dockercfg-m6pb5\"" Jul 14 12:06:05.048622 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.048606 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5q25\"/\"kube-root-ca.crt\"" Jul 14 12:06:05.048686 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.048641 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5q25\"/\"openshift-service-ca.crt\"" Jul 14 12:06:05.052171 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.052149 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4"] Jul 14 12:06:05.154170 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.154133 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-proc\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.154170 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.154173 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-podres\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.154496 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.154326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-sys\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.154496 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.154386 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq49w\" (UniqueName: \"kubernetes.io/projected/096f04d6-235a-4e58-aac1-1aabf7742643-kube-api-access-tq49w\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.154496 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.154424 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-lib-modules\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.254995 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.254955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-sys\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.254995 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.254999 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq49w\" (UniqueName: \"kubernetes.io/projected/096f04d6-235a-4e58-aac1-1aabf7742643-kube-api-access-tq49w\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255213 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-lib-modules\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255213 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255080 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-sys\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255213 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255152 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-lib-modules\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255213 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-proc\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255213 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-podres\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255213 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255206 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-proc\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.255531 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.255283 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/096f04d6-235a-4e58-aac1-1aabf7742643-podres\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.263987 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.263964 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq49w\" (UniqueName: \"kubernetes.io/projected/096f04d6-235a-4e58-aac1-1aabf7742643-kube-api-access-tq49w\") pod \"perf-node-gather-daemonset-m82j4\" (UID: \"096f04d6-235a-4e58-aac1-1aabf7742643\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.355416 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.355375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.489310 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.489155 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/2.log" Jul 14 12:06:05.490888 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.490861 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4"] Jul 14 12:06:05.492719 ip-10-0-136-255 kubenswrapper[2568]: W0714 12:06:05.492687 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod096f04d6_235a_4e58_aac1_1aabf7742643.slice/crio-78e470439d60a5aafccd6fd82aa380bfb930af1eade15e10bc79eb34e15cdb29 WatchSource:0}: Error finding container 78e470439d60a5aafccd6fd82aa380bfb930af1eade15e10bc79eb34e15cdb29: Status 404 returned error can't find the container with id 78e470439d60a5aafccd6fd82aa380bfb930af1eade15e10bc79eb34e15cdb29 Jul 14 12:06:05.498727 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.498707 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-l8hd4_090cb996-2729-46af-adac-941ef69bc0f8/console-operator/3.log" Jul 14 12:06:05.972625 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.972528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" event={"ID":"096f04d6-235a-4e58-aac1-1aabf7742643","Type":"ContainerStarted","Data":"b0c28c633154088f6d57a863722b1deb760ad373bc90b898f931d72cc8328896"} Jul 14 12:06:05.972625 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.972585 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" event={"ID":"096f04d6-235a-4e58-aac1-1aabf7742643","Type":"ContainerStarted","Data":"78e470439d60a5aafccd6fd82aa380bfb930af1eade15e10bc79eb34e15cdb29"} Jul 14 12:06:05.972860 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.972661 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:05.988715 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:05.988663 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" podStartSLOduration=0.988641916 podStartE2EDuration="988.641916ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 12:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:06:05.988386657 +0000 UTC m=+2895.844220828" watchObservedRunningTime="2026-07-14 12:06:05.988641916 +0000 UTC m=+2895.844476091" Jul 14 12:06:06.597849 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:06.597820 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-bdtbp_ac55edc5-a5d4-4283-8b76-00d0fd56ec8e/volume-data-source-validator/0.log" Jul 14 12:06:07.701924 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:07.701891 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mq2gr_53beda37-da6d-4cff-88c6-8341416a83de/dns/0.log" Jul 14 12:06:07.729559 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:07.729528 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mq2gr_53beda37-da6d-4cff-88c6-8341416a83de/kube-rbac-proxy/0.log" Jul 14 12:06:07.830604 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:07.830574 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t2jvz_445082e1-8f80-4e41-8164-2a8ce6a18b10/dns-node-resolver/0.log" Jul 14 12:06:08.459643 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:08.459608 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hxzbm_49806619-4c59-4b80-b24f-22162de9877b/node-ca/0.log" Jul 14 12:06:09.644859 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:09.644829 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-9b5zc_fa33eac3-211c-4b2f-a40b-04564300efd9/kube-auth-proxy/0.log" Jul 14 12:06:09.819471 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:09.819436 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qprhx_bb482966-a541-4ffb-a8d9-3745dad2fde3/payload-pre-processing/0.log" Jul 14 12:06:09.874176 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:09.874144 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-rpmgm_ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c/payload-processing/0.log" Jul 14 12:06:09.911831 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:09.911735 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794fd9f96d-dt6lg_00628d94-4a03-456d-8c41-0ab813236ad1/router/0.log" Jul 14 12:06:10.695831 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:10.695801 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9jdh_648326eb-7168-4bdb-a3e0-e7b510948491/serve-healthcheck-canary/0.log" Jul 14 12:06:11.251337 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:11.251305 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-wl2m9_8ca8083f-bc69-461c-80e3-d5abc98eb611/insights-operator/0.log" Jul 14 12:06:11.255791 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:11.255766 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-wl2m9_8ca8083f-bc69-461c-80e3-d5abc98eb611/insights-operator/1.log" Jul 14 12:06:11.441432 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:11.441400 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5j9x_6a2a6105-4b48-40b0-8f75-2023d94157fa/kube-rbac-proxy/0.log" Jul 14 12:06:11.463221 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:11.463193 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5j9x_6a2a6105-4b48-40b0-8f75-2023d94157fa/exporter/0.log" Jul 14 12:06:11.490273 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:11.490232 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5j9x_6a2a6105-4b48-40b0-8f75-2023d94157fa/extractor/0.log" Jul 14 12:06:11.987916 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:11.987887 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-m82j4" Jul 14 12:06:13.863480 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:13.863446 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vp8rc_76879cef-26ef-4c81-adef-9fa7a7fa7a43/manager/0.log" Jul 14 12:06:14.107185 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:14.107139 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-q5dc9_94c6e106-e6c7-47ea-8682-f1c54fc2d665/manager/0.log" Jul 14 12:06:14.235033 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:14.235002 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fsrww_278a93c6-7346-469e-a8e2-b5f7abc77f11/server/0.log" Jul 14 12:06:14.350532 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:14.350497 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d2jfx_08aba400-78d1-44be-8921-7112772a70aa/manager/0.log" Jul 14 12:06:14.588065 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:14.588012 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-zwpqd_e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1/manager/0.log" Jul 14 12:06:14.608251 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:14.608226 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-zwpqd_e19a6ce3-faf7-4a3d-bdbf-0c2008f3b5c1/copy-manifests/0.log" Jul 14 12:06:15.334704 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:15.334674 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-9b5zc_fa33eac3-211c-4b2f-a40b-04564300efd9/kube-auth-proxy/0.log" Jul 14 12:06:15.425198 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:15.425104 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qprhx_bb482966-a541-4ffb-a8d9-3745dad2fde3/payload-pre-processing/0.log" Jul 14 12:06:15.451974 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:15.451939 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-rpmgm_ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c/payload-processing/0.log" Jul 14 12:06:15.470623 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:15.470594 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794fd9f96d-dt6lg_00628d94-4a03-456d-8c41-0ab813236ad1/router/0.log" Jul 14 12:06:16.853366 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:16.853335 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-9b5zc_fa33eac3-211c-4b2f-a40b-04564300efd9/kube-auth-proxy/0.log" Jul 14 12:06:16.952504 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:16.952466 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qprhx_bb482966-a541-4ffb-a8d9-3745dad2fde3/payload-pre-processing/0.log" Jul 14 12:06:16.979290 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:16.979237 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-rpmgm_ac13aa1f-5d9a-45c5-95e0-4ee3d06d084c/payload-processing/0.log" Jul 14 12:06:16.993802 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:16.993776 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794fd9f96d-dt6lg_00628d94-4a03-456d-8c41-0ab813236ad1/router/0.log" Jul 14 12:06:20.900471 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:20.900440 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-qfbpl_0ce35024-2bd2-4602-ade2-222d50edcfb1/migrator/0.log" Jul 14 12:06:20.923635 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:20.923603 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-qfbpl_0ce35024-2bd2-4602-ade2-222d50edcfb1/graceful-termination/0.log" Jul 14 12:06:21.314951 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:21.314915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-l2cns_c6eb804d-f51c-45b4-a39d-e56acbe6d53f/kube-storage-version-migrator-operator/1.log" Jul 14 12:06:21.316419 ip-10-0-136-255 kubenswrapper[2568]: I0714 12:06:21.316395 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-l2cns_c6eb804d-f51c-45b4-a39d-e56acbe6d53f/kube-storage-version-migrator-operator/0.log"