Jul 15 18:58:50.730994 ip-10-0-137-168 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 18:58:50.731007 ip-10-0-137-168 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 18:58:50.731018 ip-10-0-137-168 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 18:58:50.731363 ip-10-0-137-168 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 18:59:00.982532 ip-10-0-137-168 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 18:59:00.982549 ip-10-0-137-168 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2e08bd425031445e92326c654496919f -- Jul 15 19:00:30.399518 ip-10-0-137-168 systemd[1]: Starting Kubernetes Kubelet... Jul 15 19:00:30.860120 ip-10-0-137-168 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:00:30.860120 ip-10-0-137-168 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 19:00:30.860120 ip-10-0-137-168 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:00:30.860120 ip-10-0-137-168 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 19:00:30.860120 ip-10-0-137-168 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:00:30.860736 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.860213 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 19:00:30.866985 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866967 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:00:30.866985 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866983 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:00:30.866985 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866987 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866990 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866993 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866996 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.866999 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867002 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867005 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867007 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867010 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867013 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867016 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867024 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867027 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867030 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867032 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867035 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867038 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867040 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867042 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:00:30.867078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867045 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867049 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867053 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867056 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867059 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867062 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867064 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867068 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867071 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867073 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867076 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867079 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867082 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867085 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867088 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867090 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867093 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867095 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867098 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:00:30.867538 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867101 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867103 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867106 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867108 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867111 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867114 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867116 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867119 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867122 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867124 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867127 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867129 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867132 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867134 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867137 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867141 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867143 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867146 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867149 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867154 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:00:30.868000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867157 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867160 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867164 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867166 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867169 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867172 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867175 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867177 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867180 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867182 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867185 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867188 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867191 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867193 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867195 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867198 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867200 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867203 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867206 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867208 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:00:30.868517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867211 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867213 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867215 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867218 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867220 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867223 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867226 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867241 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867245 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867247 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867250 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867253 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867255 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867258 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867260 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867263 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867266 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867268 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867271 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867970 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867977 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:00:30.868963 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867979 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867982 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867985 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867988 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867990 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867993 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.867996 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868001 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868005 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868008 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868012 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868015 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868018 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868022 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868025 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868028 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868031 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868033 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868036 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:00:30.869496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868039 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868042 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868044 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868047 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868050 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868052 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868054 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868057 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868060 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868063 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868066 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868068 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868070 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868073 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868075 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868078 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868081 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868090 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868093 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868095 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:00:30.869956 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868098 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868100 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868103 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868105 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868108 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868111 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868113 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868116 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868118 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868121 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868123 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868125 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868129 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868132 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868134 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868137 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868139 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868142 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868144 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868147 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:00:30.870495 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868149 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868152 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868154 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868157 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868160 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868162 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868165 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868167 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868169 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868172 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868179 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868182 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868185 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868187 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868190 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868192 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868195 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868198 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868200 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:00:30.870961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868203 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868207 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868210 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868212 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868214 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868217 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868219 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868222 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868224 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868226 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868229 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868248 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868251 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868254 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868256 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868259 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868262 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868264 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868267 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868679 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:00:30.871426 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868684 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868688 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868691 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868694 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868697 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868700 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868702 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868705 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868708 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868710 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868713 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868715 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868718 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868721 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868723 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868725 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868728 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868730 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868733 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868735 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:00:30.871906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868738 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868740 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868743 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868745 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868747 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868750 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868752 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868755 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868757 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868760 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868762 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868765 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868767 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868770 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868772 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868775 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868778 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868780 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868783 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:00:30.872402 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868786 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868789 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868791 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868794 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868797 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868799 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868802 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868804 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868806 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868809 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868812 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868814 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868817 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868823 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868825 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868827 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868830 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868832 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868836 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868840 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:00:30.872846 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868843 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868846 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868849 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868851 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868854 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868858 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868862 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868866 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868869 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868872 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868880 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868883 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868886 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868889 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868892 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868895 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868897 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868900 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868903 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:00:30.873325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868905 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868908 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868911 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868913 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868915 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868918 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868922 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868924 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868927 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868930 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868933 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868935 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868938 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868940 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868943 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868945 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868948 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868951 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868953 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.868956 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869020 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 19:00:30.873785 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869027 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869035 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869040 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869044 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869047 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869051 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869056 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869059 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869061 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869065 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869068 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869071 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869074 2572 flags.go:64] FLAG: --cgroup-root="" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869077 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869080 2572 flags.go:64] FLAG: --client-ca-file="" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869083 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869086 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869092 2572 flags.go:64] FLAG: --cluster-domain="" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869095 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869098 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869101 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869104 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869108 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 19:00:30.874336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869111 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869115 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869118 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869121 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869124 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869127 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869130 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869133 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869137 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869140 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869143 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869146 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869149 2572 flags.go:64] FLAG: --enable-server="true" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869152 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869157 2572 flags.go:64] FLAG: --event-burst="100" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869160 2572 flags.go:64] FLAG: --event-qps="50" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869163 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869165 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869168 2572 flags.go:64] FLAG: --eviction-hard="" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869172 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869175 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869177 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869180 2572 flags.go:64] FLAG: --eviction-soft="" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869183 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869186 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 19:00:30.874868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869189 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869193 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869195 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869198 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869201 2572 flags.go:64] FLAG: --feature-gates="" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869205 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869207 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869211 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869215 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869218 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869221 2572 flags.go:64] FLAG: --help="false" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869224 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-137-168.ec2.internal" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869227 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869245 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869248 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869252 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869255 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869258 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869261 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869264 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869266 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869270 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869273 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869276 2572 flags.go:64] FLAG: --kube-reserved="" Jul 15 19:00:30.875524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869279 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869282 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869285 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869287 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869290 2572 flags.go:64] FLAG: --lock-file="" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869293 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869296 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869299 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869304 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869309 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869311 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869314 2572 flags.go:64] FLAG: --logging-format="text" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869317 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869321 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869323 2572 flags.go:64] FLAG: --manifest-url="" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869326 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869331 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869334 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869339 2572 flags.go:64] FLAG: --max-pods="110" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869341 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869344 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869347 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869350 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869353 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869356 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 19:00:30.876135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869359 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869366 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869369 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869372 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869375 2572 flags.go:64] FLAG: --pod-cidr="" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869378 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869384 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869387 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869390 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869393 2572 flags.go:64] FLAG: --port="10250" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869396 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869399 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09cf2eb9e7deca6fa" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869402 2572 flags.go:64] FLAG: --qos-reserved="" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869405 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869408 2572 flags.go:64] FLAG: --register-node="true" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869411 2572 flags.go:64] FLAG: --register-with-taints="" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869416 2572 flags.go:64] FLAG: --registry-burst="10" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869419 2572 flags.go:64] FLAG: --registry-qps="5" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869421 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869424 2572 flags.go:64] FLAG: --reserved-memory="" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869428 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869431 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869434 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869437 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869441 2572 flags.go:64] FLAG: --runonce="false" Jul 15 19:00:30.876774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869444 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869447 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869450 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869452 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869455 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869458 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869461 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869464 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869467 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869470 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869473 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869475 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869478 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869482 2572 flags.go:64] FLAG: --system-cgroups="" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869485 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869490 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869493 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869495 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869500 2572 flags.go:64] FLAG: --tls-min-version="" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869503 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869506 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869509 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869512 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869516 2572 flags.go:64] FLAG: --v="2" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869520 2572 flags.go:64] FLAG: --version="false" Jul 15 19:00:30.877434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869523 2572 flags.go:64] FLAG: --vmodule="" Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869527 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.869530 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869655 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869660 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869663 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869670 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869672 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869675 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869677 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869680 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869683 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869685 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869688 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869690 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869693 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869695 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869698 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869701 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:00:30.878026 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869703 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869706 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869709 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869711 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869714 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869717 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869719 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869722 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869724 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869727 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869729 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869732 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869735 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869737 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869740 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869742 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869745 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869747 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869750 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869754 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:00:30.878507 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869757 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869759 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869762 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869764 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869767 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869769 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869772 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869774 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869777 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869780 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869782 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869785 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869787 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869790 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869792 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869795 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869797 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869800 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869803 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:00:30.879058 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869806 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869810 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869814 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869818 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869821 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869824 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869827 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869829 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869832 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869834 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869837 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869840 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869843 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869846 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869849 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869851 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869854 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869856 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869859 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869861 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:00:30.879827 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869864 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869866 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869869 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869871 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869874 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869877 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869879 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869882 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869884 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869888 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869891 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869894 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869896 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869899 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869901 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869904 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869906 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869909 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869911 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869913 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:00:30.880445 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869916 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869918 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869921 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.869923 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.870754 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.877754 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.877767 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.878525 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.878536 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878598 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878603 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878606 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878610 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878613 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878616 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:00:30.880929 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878620 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878623 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878626 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878629 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878632 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878635 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878638 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878641 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878644 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878647 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878649 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878652 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878654 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878657 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878659 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878662 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878665 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878667 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878671 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:00:30.881383 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878675 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878679 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878682 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878685 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878688 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878691 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878694 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878696 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878699 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878702 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878704 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878709 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878712 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878715 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878718 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878720 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878723 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878726 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878729 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:00:30.881839 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878731 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878734 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878737 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878739 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878742 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878744 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878747 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878749 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878752 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878754 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878757 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878759 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878763 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878766 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878769 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878772 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878775 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878778 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878781 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:00:30.882325 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878784 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878787 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878789 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878792 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878794 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878797 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878799 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878802 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878805 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878807 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878810 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878812 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878815 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878817 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878820 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878858 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878904 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878907 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878910 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878913 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878916 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:00:30.882780 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878919 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878922 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878925 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878928 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878930 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878933 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878936 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878939 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878945 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878948 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878951 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878954 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878957 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878959 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:00:30.883326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.878962 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.878968 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880289 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880301 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880305 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880308 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880311 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880313 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880316 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880319 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880322 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880324 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880328 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880335 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:00:30.883674 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880338 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880342 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880344 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880347 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880349 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880352 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880354 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880357 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880360 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880362 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880366 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880368 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880371 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880374 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880378 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880382 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880385 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880387 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880390 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880392 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:00:30.884023 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880395 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880398 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880401 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880403 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880407 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880409 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880411 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880414 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880416 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880419 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880422 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880424 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880429 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880433 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880436 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880441 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880443 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880447 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880450 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880454 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:00:30.884519 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880457 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880461 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880465 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880467 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880470 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880473 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880476 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880479 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880481 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880484 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880486 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880491 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880494 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880497 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880499 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880502 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880505 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880508 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880510 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:00:30.885091 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880514 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880518 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880520 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880523 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880526 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880529 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880534 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880537 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880540 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880542 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880545 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880548 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880551 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880554 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880557 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880560 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880562 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880565 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880568 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880570 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:00:30.885961 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880573 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880575 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880578 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880582 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880584 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880587 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880590 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:30.880592 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.880597 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.880713 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.882849 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.884441 2572 server.go:1019] "Starting client certificate rotation" Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.884549 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:00:30.886546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.885118 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:00:30.887830 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.887815 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:00:30.890608 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.890590 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:00:30.904698 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.904683 2572 log.go:25] "Validated CRI v1 runtime API" Jul 15 19:00:30.907652 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.907637 2572 log.go:25] "Validated CRI v1 image API" Jul 15 19:00:30.909909 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.909893 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 19:00:30.914928 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.914908 2572 fs.go:135] Filesystem UUIDs: map[65427409-3dfa-47d0-91fd-4fe4e16e6879:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e40f0a10-ada4-4d44-86af-2006878b8cd8:/dev/nvme0n1p3] Jul 15 19:00:30.915015 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.914927 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 19:00:30.915198 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.915180 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:00:30.919899 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.919793 2572 manager.go:216] Machine: {Timestamp:2026-07-15 19:00:30.918490798 +0000 UTC m=+0.385465100 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091475 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25ba21bfdf634e308122c9976e6ae5 SystemUUID:ec25ba21-bfdf-634e-3081-22c9976e6ae5 BootID:2e08bd42-5031-445e-9232-6c654496919f Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e3:2e:5f:ca:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e3:2e:5f:ca:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:d3:f3:d5:62:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 19:00:30.919899 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.919889 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 19:00:30.920034 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.919969 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 19:00:30.922982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.922949 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 19:00:30.923120 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.922986 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 19:00:30.924593 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.924583 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 19:00:30.924630 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.924596 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 19:00:30.924630 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.924603 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:00:30.925258 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.925248 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:00:30.926587 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.926577 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 19:00:30.931840 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.931828 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:00:30.931969 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.931960 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 19:00:30.931999 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.931991 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 15 19:00:30.932031 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.932005 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 19:00:30.932031 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.932023 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 19:00:30.932111 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.932036 2572 kubelet.go:398] "Adding apiserver pod source" Jul 15 19:00:30.932111 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.932048 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 19:00:30.933795 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.933771 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:00:30.933893 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.933800 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:00:30.936776 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.936758 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 19:00:30.938697 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938680 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 19:00:30.938777 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938712 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 19:00:30.938872 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938862 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 19:00:30.938911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938877 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 19:00:30.938911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938883 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 19:00:30.938911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938889 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 19:00:30.938911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938897 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 19:00:30.938911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.938903 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 19:00:30.939416 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.939407 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 19:00:30.939416 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.939417 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 19:00:30.939478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.939423 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 19:00:30.939478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.939429 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 19:00:30.939478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.939437 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 19:00:30.939478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.939445 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 19:00:30.941463 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.941450 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 19:00:30.941463 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.941459 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 19:00:30.943325 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.943307 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 19:00:30.943425 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.943357 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 19:00:30.943425 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.943356 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 19:00:30.944564 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.944552 2572 server.go:1268] "Started kubelet" Jul 15 19:00:30.944627 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.944607 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 19:00:30.944674 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.944640 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 19:00:30.944718 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.944692 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 19:00:30.945370 ip-10-0-137-168 systemd[1]: Started Kubernetes Kubelet. Jul 15 19:00:30.945946 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.945926 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 19:00:30.946781 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.946761 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 15 19:00:30.955207 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.955186 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 19:00:30.955800 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.955758 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 19:00:30.955800 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.955758 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 19:00:30.955948 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.955873 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:30.956599 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956582 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 19:00:30.956732 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956597 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 19:00:30.956732 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956734 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 19:00:30.956869 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956673 2572 factory.go:55] Registering systemd factory Jul 15 19:00:30.956869 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956762 2572 factory.go:223] Registration of the systemd container factory successfully Jul 15 19:00:30.956869 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956837 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 15 19:00:30.956869 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.956844 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 15 19:00:30.957051 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.957036 2572 factory.go:153] Registering CRI-O factory Jul 15 19:00:30.957119 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.957056 2572 factory.go:223] Registration of the crio container factory successfully Jul 15 19:00:30.957148 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.957122 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 19:00:30.957180 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.957149 2572 factory.go:103] Registering Raw factory Jul 15 19:00:30.957180 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.957164 2572 manager.go:1196] Started watching for new ooms in manager Jul 15 19:00:30.957585 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.957571 2572 manager.go:349] Starting recovery of all containers Jul 15 19:00:30.959736 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.959711 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 19:00:30.959895 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.959876 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 19:00:30.960934 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.959874 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb96f4adc75 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.944525429 +0000 UTC m=+0.411499731,LastTimestamp:2026-07-15 19:00:30.944525429 +0000 UTC m=+0.411499731,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:30.965122 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.965090 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 19:00:30.970087 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.970046 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n4gbd" Jul 15 19:00:30.971056 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.971034 2572 manager.go:354] Recovery completed Jul 15 19:00:30.976675 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.976663 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:30.977437 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.977420 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n4gbd" Jul 15 19:00:30.979011 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.978990 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:30.979102 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.979026 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:30.979102 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.979040 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:30.979615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.979599 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 19:00:30.979615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.979613 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 19:00:30.979615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.979615 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:30.979763 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.979631 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:00:30.980395 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.980378 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:30.980464 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.980408 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:30.980464 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:30.980418 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:30.981541 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.981464 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb971590ea0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.979010208 +0000 UTC m=+0.445984511,LastTimestamp:2026-07-15 19:00:30.979010208 +0000 UTC m=+0.445984511,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:30.989518 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.989446 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb97159675c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.979032924 +0000 UTC m=+0.446007228,LastTimestamp:2026-07-15 19:00:30.979032924 +0000 UTC m=+0.446007228,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:30.999072 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:30.999002 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb9715999b8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.979045816 +0000 UTC m=+0.446020119,LastTimestamp:2026-07-15 19:00:30.979045816 +0000 UTC m=+0.446020119,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:31.010003 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.009912 2572 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-168.ec2.internal.18c28bb971590ea0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb971590ea0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.979010208 +0000 UTC m=+0.445984511,LastTimestamp:2026-07-15 19:00:30.980394964 +0000 UTC m=+0.447369266,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:31.018713 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.018651 2572 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-168.ec2.internal.18c28bb97159675c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb97159675c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.979032924 +0000 UTC m=+0.446007228,LastTimestamp:2026-07-15 19:00:30.980413646 +0000 UTC m=+0.447387948,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.027036 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.027069 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.027087 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.027116 2572 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.027819 2572 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-168.ec2.internal.18c28bb9715999b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c28bb9715999b8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-15 19:00:30.979045816 +0000 UTC m=+0.446020119,LastTimestamp:2026-07-15 19:00:30.980422405 +0000 UTC m=+0.447396707,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.037521 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.056702 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.117981 2572 policy_none.go:49] "None policy: Start" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.118000 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 19:00:31.118583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.118012 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 19:00:31.119296 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.119148 2572 policy_none.go:47] "Start" Jul 15 19:00:31.127867 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.127848 2572 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 15 19:00:31.153751 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.153738 2572 manager.go:337] "Starting Device Plugin manager" Jul 15 19:00:31.153850 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.153766 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 19:00:31.153850 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.153776 2572 server.go:87] "Starting device plugin registration server" Jul 15 19:00:31.154010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.153995 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 19:00:31.154315 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.154010 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 19:00:31.154315 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.154121 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 19:00:31.154315 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.154207 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 19:00:31.154315 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.154216 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 19:00:31.155345 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.155330 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 19:00:31.155416 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.155369 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.168669 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.168649 2572 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-168.ec2.internal\" not found" node="ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.254796 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.254771 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:31.255775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.255760 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:31.255858 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.255788 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:31.255858 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.255799 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:31.255858 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.255822 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.264535 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.264518 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.264600 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.264539 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-168.ec2.internal\": node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.292498 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.292466 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.328692 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.328664 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal"] Jul 15 19:00:31.328773 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.328735 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:31.329534 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.329520 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:31.329602 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.329546 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:31.329602 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.329556 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:31.331140 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331127 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:31.331269 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331254 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.331332 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331298 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:31.331810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331786 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:31.331810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331796 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:31.331951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331818 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:31.331951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331821 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:31.331951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331828 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:31.331951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.331837 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:31.332971 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.332954 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.333018 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.332984 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:00:31.333633 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.333618 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:00:31.333719 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.333642 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:00:31.333719 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.333651 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:00:31.353863 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.353842 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-168.ec2.internal\" not found" node="ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.358129 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.358114 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-168.ec2.internal\" not found" node="ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.359175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.359159 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/631442f90a9aec34b0d279ea0f5f9c73-config\") pod \"kube-apiserver-proxy-ip-10-0-137-168.ec2.internal\" (UID: \"631442f90a9aec34b0d279ea0f5f9c73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.359242 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.359184 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.359242 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.359203 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.392930 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.392874 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.459513 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.459491 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/631442f90a9aec34b0d279ea0f5f9c73-config\") pod \"kube-apiserver-proxy-ip-10-0-137-168.ec2.internal\" (UID: \"631442f90a9aec34b0d279ea0f5f9c73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.459638 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.459519 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.459638 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.459539 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.459638 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.459582 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/631442f90a9aec34b0d279ea0f5f9c73-config\") pod \"kube-apiserver-proxy-ip-10-0-137-168.ec2.internal\" (UID: \"631442f90a9aec34b0d279ea0f5f9c73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.459638 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.459594 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.459638 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.459637 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.493634 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.493606 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.594921 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.594880 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.660096 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.660030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.662433 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.662416 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.695530 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.695497 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.798521 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.798468 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 15 19:00:31.813309 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.813244 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:00:31.856580 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.856547 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.868018 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.867994 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:00:31.870049 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.870030 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.883241 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.883213 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 19:00:31.883416 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.883367 2572 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.137.168:42912->98.83.60.117:6443: use of closed network connection" Jul 15 19:00:31.883416 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.883384 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:00:31.883508 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.883415 2572 kubelet.go:3250] "Failed creating a mirror pod" err="unexpected error when reading response body. Please retry. Original error: read tcp 10.0.137.168:42912->98.83.60.117:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 15 19:00:31.934056 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.933984 2572 apiserver.go:52] "Watching apiserver" Jul 15 19:00:31.951438 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.951415 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:00:31.953813 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.953794 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 19:00:31.954951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.954930 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-kt4sp","openshift-ovn-kubernetes/ovnkube-node-fn4z6","kube-system/global-pull-secret-syncer-b4dg5","kube-system/konnectivity-agent-8xkv8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4","openshift-cluster-node-tuning-operator/tuned-vjvvz","openshift-dns/node-resolver-2npnt","openshift-multus/multus-additional-cni-plugins-vb4xn","openshift-network-diagnostics/network-check-target-2bmgw","openshift-network-operator/iptables-alerter-767bx","kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal","openshift-image-registry/node-ca-dz2bk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal","openshift-multus/multus-nmxwn"] Jul 15 19:00:31.955878 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.955863 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 19:00:31.957240 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.957058 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:31.957240 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.957189 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:31.959178 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.959156 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.960503 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.960486 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:31.960584 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.960544 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:31.961721 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961697 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-kubelet\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.961836 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-systemd-units\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.961836 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961757 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-systemd\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.961836 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961808 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-cni-bin\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961849 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovn-node-metrics-cert\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961875 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-slash\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961898 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-log-socket\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961921 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-run-ovn-kubernetes\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961930 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961945 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovnkube-config\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961970 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-284zk\" (UniqueName: \"kubernetes.io/projected/7bdd365b-6d8b-4b34-b68f-0d17f9600606-kube-api-access-284zk\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.961995 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962018 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-run-netns\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-node-log\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962063 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-cni-netd\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962055 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9v5f\" (UniqueName: \"kubernetes.io/projected/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-kube-api-access-l9v5f\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962110 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-etc-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962131 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-ovn\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962157 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962181 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-env-overrides\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovnkube-script-lib\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-var-lib-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962253 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962109 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.962609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.962254 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:31.963365 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.963347 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 19:00:31.963443 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.963373 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 19:00:31.963443 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.963415 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 19:00:31.963568 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.963440 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dcldl\"" Jul 15 19:00:31.963630 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.963614 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:31.965042 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.965026 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5hg4v\"" Jul 15 19:00:31.965210 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.965194 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 19:00:31.965324 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.965296 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:31.966928 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.966907 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 19:00:31.967039 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.966957 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:31.967039 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.966983 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 19:00:31.967039 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.967003 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.967039 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.967012 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.967487 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.967471 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v6bb8\"" Jul 15 19:00:31.967714 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.967699 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.968202 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.968187 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x8tls\"" Jul 15 19:00:31.968202 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.968197 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.968580 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.968565 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:31.969644 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.969627 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.969737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.969723 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h66gz\"" Jul 15 19:00:31.970001 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.969985 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:31.970116 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:31.970093 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:31.970167 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.970127 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:00:31.970205 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.970175 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.971152 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971139 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 19:00:31.971380 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971365 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.971627 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971612 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 19:00:31.971627 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971620 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.971759 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971677 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m75z5\"" Jul 15 19:00:31.971759 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971693 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 19:00:31.971951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.971935 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:31.973731 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.973714 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:31.975185 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.975166 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nmxwn" Jul 15 19:00:31.977115 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.976990 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.977115 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.977054 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 19:00:31.977115 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.977075 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 19:00:31.977115 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.977098 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.977435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.977172 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6hxs6\"" Jul 15 19:00:31.978577 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.978564 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 19:00:31.978655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.978640 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 19:00:31.978730 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.978639 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 19:00:31.978846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.978750 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j2pz7\"" Jul 15 19:00:31.978846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.978713 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zbq2x\"" Jul 15 19:00:31.993709 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:31.993692 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cgbgm" Jul 15 19:00:32.000537 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.000513 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cgbgm" Jul 15 19:00:32.033135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.033118 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:00:32.057950 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.057932 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 19:00:32.062980 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.062957 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-tuned\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.063090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.062991 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cni-binary-copy\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.063090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063011 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-kubelet\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063037 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-systemd\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-cnibin\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.063341 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063164 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-systemd\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063341 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063203 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-kubelet\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063341 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063226 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.063341 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6196293c-156c-4f46-9f14-f228ec4b2faa-hosts-file\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063325 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-device-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063379 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6196293c-156c-4f46-9f14-f228ec4b2faa-tmp-dir\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063408 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-node-log\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063441 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-cni-netd\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063460 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwcvn\" (UniqueName: \"kubernetes.io/projected/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-kube-api-access-nwcvn\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063484 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysconfig\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-ovn\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063510 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-cni-netd\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.063506 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063516 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-node-log\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.063547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063551 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-ovn\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063556 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-system-cni-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.063580 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:32.563561526 +0000 UTC m=+2.030535837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063631 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-284zk\" (UniqueName: \"kubernetes.io/projected/7bdd365b-6d8b-4b34-b68f-0d17f9600606-kube-api-access-284zk\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-cni-bin\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063684 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-iptables-alerter-script\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063701 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-host-slash\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063723 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-socket-dir-parent\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063750 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj7rp\" (UniqueName: \"kubernetes.io/projected/5e017ac5-e1f6-4735-b095-7db2d6e18559-kube-api-access-zj7rp\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063770 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-run\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063787 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dda6c0d0-323a-4782-8f94-95dcb580be7e-tmp\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063825 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqqrn\" (UniqueName: \"kubernetes.io/projected/6196293c-156c-4f46-9f14-f228ec4b2faa-kube-api-access-cqqrn\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063844 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-os-release\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063860 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfbd346b-c505-44fb-828c-0fc38124c540-konnectivity-ca\") pod \"konnectivity-agent-8xkv8\" (UID: \"cfbd346b-c505-44fb-828c-0fc38124c540\") " pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063906 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e017ac5-e1f6-4735-b095-7db2d6e18559-cni-binary-copy\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063927 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063942 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063959 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f2n4\" (UniqueName: \"kubernetes.io/projected/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-kube-api-access-8f2n4\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.063984 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-854gc\" (UniqueName: \"kubernetes.io/projected/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-kube-api-access-854gc\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064021 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-env-overrides\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064049 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-multus-certs\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064068 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064102 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-var-lib-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064129 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-systemd-units\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064145 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-cni-bin\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064192 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-cni-bin\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064230 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-systemd-units\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064264 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-netns\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064270 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-var-lib-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064294 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-conf-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064311 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x8bh\" (UniqueName: \"kubernetes.io/projected/dda6c0d0-323a-4782-8f94-95dcb580be7e-kube-api-access-7x8bh\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064332 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-slash\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.064885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064358 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovnkube-config\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064385 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-slash\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-os-release\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064437 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-daemon-config\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064454 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-etc-kubernetes\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064470 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-sys\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064487 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-var-lib-kubelet\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064513 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-system-cni-dir\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfbd346b-c505-44fb-828c-0fc38124c540-agent-certs\") pod \"konnectivity-agent-8xkv8\" (UID: \"cfbd346b-c505-44fb-828c-0fc38124c540\") " pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064544 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-cni-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064558 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064594 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-registration-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064621 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-systemd\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064655 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-etc-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064697 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-socket-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064718 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtv8q\" (UniqueName: \"kubernetes.io/projected/40039ba0-8699-446d-bbcb-270bed4a7beb-kube-api-access-vtv8q\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064748 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysctl-conf\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.065582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064753 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-etc-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064773 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-host\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-serviceca\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovn-node-metrics-cert\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064884 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-modprobe-d\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064902 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-run-openvswitch\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064908 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-host\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064934 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-log-socket\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064960 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-log-socket\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064960 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-run-ovn-kubernetes\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.064987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-cni-multus\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065004 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-hostroot\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065006 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-run-ovn-kubernetes\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-kubernetes\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-run-netns\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065074 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-k8s-cni-cncf-io\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.066228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065085 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-run-netns\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065089 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-kubelet\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065120 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-etc-selinux\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065137 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysctl-d\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065151 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-lib-modules\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065166 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cnibin\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065184 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9v5f\" (UniqueName: \"kubernetes.io/projected/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-kube-api-access-l9v5f\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065208 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065249 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdd365b-6d8b-4b34-b68f-0d17f9600606-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovnkube-script-lib\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065355 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/35b4663c-ea09-4be7-a31c-660e76c46799-kubelet-config\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/35b4663c-ea09-4be7-a31c-660e76c46799-dbus\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.065413 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-sys-fs\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.066214 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.066392 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-env-overrides\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.066440 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovnkube-config\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.066843 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.066518 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovnkube-script-lib\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.069908 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.069890 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bdd365b-6d8b-4b34-b68f-0d17f9600606-ovn-node-metrics-cert\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.082019 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.081992 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9v5f\" (UniqueName: \"kubernetes.io/projected/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-kube-api-access-l9v5f\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:32.082335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.082315 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-284zk\" (UniqueName: \"kubernetes.io/projected/7bdd365b-6d8b-4b34-b68f-0d17f9600606-kube-api-access-284zk\") pod \"ovnkube-node-fn4z6\" (UID: \"7bdd365b-6d8b-4b34-b68f-0d17f9600606\") " pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.120185 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.120134 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 18:55:31 +0000 UTC" deadline="2028-02-10 23:10:06.189348204 +0000 UTC" Jul 15 19:00:32.120185 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.120182 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13804h9m34.069168695s" Jul 15 19:00:32.165687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165657 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysconfig\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.165687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-system-cni-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165710 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-cni-bin\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-iptables-alerter-script\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165796 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-system-cni-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165803 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-cni-bin\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165821 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysconfig\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-host-slash\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165873 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-socket-dir-parent\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165881 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-host-slash\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj7rp\" (UniqueName: \"kubernetes.io/projected/5e017ac5-e1f6-4735-b095-7db2d6e18559-kube-api-access-zj7rp\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165927 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-run\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165944 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-socket-dir-parent\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.165948 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165949 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dda6c0d0-323a-4782-8f94-95dcb580be7e-tmp\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.165976 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqqrn\" (UniqueName: \"kubernetes.io/projected/6196293c-156c-4f46-9f14-f228ec4b2faa-kube-api-access-cqqrn\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166036 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-run\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166066 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-os-release\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166113 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfbd346b-c505-44fb-828c-0fc38124c540-konnectivity-ca\") pod \"konnectivity-agent-8xkv8\" (UID: \"cfbd346b-c505-44fb-828c-0fc38124c540\") " pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166140 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166190 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e017ac5-e1f6-4735-b095-7db2d6e18559-cni-binary-copy\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166218 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166263 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166295 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f2n4\" (UniqueName: \"kubernetes.io/projected/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-kube-api-access-8f2n4\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166324 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-854gc\" (UniqueName: \"kubernetes.io/projected/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-kube-api-access-854gc\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-multus-certs\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166397 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-iptables-alerter-script\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166416 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-netns\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-conf-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166472 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x8bh\" (UniqueName: \"kubernetes.io/projected/dda6c0d0-323a-4782-8f94-95dcb580be7e-kube-api-access-7x8bh\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.166574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-os-release\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166528 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-daemon-config\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166554 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-etc-kubernetes\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166578 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-sys\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-var-lib-kubelet\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-system-cni-dir\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfbd346b-c505-44fb-828c-0fc38124c540-agent-certs\") pod \"konnectivity-agent-8xkv8\" (UID: \"cfbd346b-c505-44fb-828c-0fc38124c540\") " pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166686 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-cni-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166713 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166741 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-registration-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-systemd\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-socket-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166816 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfbd346b-c505-44fb-828c-0fc38124c540-konnectivity-ca\") pod \"konnectivity-agent-8xkv8\" (UID: \"cfbd346b-c505-44fb-828c-0fc38124c540\") " pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166824 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtv8q\" (UniqueName: \"kubernetes.io/projected/40039ba0-8699-446d-bbcb-270bed4a7beb-kube-api-access-vtv8q\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166850 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysctl-conf\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166874 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-host\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166897 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-sys\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.167486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166330 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-os-release\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166890 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-serviceca\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166965 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-modprobe-d\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166983 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-host\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166994 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-etc-kubernetes\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-cni-multus\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167010 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-os-release\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167021 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-hostroot\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167051 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-hostroot\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.166923 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e017ac5-e1f6-4735-b095-7db2d6e18559-cni-binary-copy\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167059 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-var-lib-kubelet\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167093 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-kubernetes\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167110 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-system-cni-dir\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167346 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-multus-certs\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167389 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-netns\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167400 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-serviceca\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167406 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-conf-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.168436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167451 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-host\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167470 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-systemd\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167544 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-daemon-config\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-kubernetes\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167573 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-socket-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167595 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-k8s-cni-cncf-io\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167603 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-cni-multus\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167546 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-modprobe-d\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167640 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-registration-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167659 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-host\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167705 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167718 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167721 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysctl-conf\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167725 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-multus-cni-dir\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-kubelet\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167770 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-var-lib-kubelet\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167775 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-etc-selinux\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.169397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167796 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-host-run-k8s-cni-cncf-io\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167804 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysctl-d\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-lib-modules\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cnibin\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167890 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/35b4663c-ea09-4be7-a31c-660e76c46799-kubelet-config\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167923 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cnibin\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167939 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/35b4663c-ea09-4be7-a31c-660e76c46799-kubelet-config\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167940 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/35b4663c-ea09-4be7-a31c-660e76c46799-dbus\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167974 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-sys-fs\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.167992 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/35b4663c-ea09-4be7-a31c-660e76c46799-dbus\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168000 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-tuned\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168023 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-lib-modules\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cni-binary-copy\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168055 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-sys-fs\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168065 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-etc-selinux\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168110 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-cnibin\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168113 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-sysctl-d\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.170215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e017ac5-e1f6-4735-b095-7db2d6e18559-cnibin\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168133 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6196293c-156c-4f46-9f14-f228ec4b2faa-hosts-file\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-device-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6196293c-156c-4f46-9f14-f228ec4b2faa-tmp-dir\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwcvn\" (UniqueName: \"kubernetes.io/projected/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-kube-api-access-nwcvn\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.168912 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dda6c0d0-323a-4782-8f94-95dcb580be7e-tmp\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.169048 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6196293c-156c-4f46-9f14-f228ec4b2faa-hosts-file\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.169157 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40039ba0-8699-446d-bbcb-270bed4a7beb-device-dir\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.169167 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.169235 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:32.669214842 +0000 UTC m=+2.136189138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.169165 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-cni-binary-copy\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.169485 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6196293c-156c-4f46-9f14-f228ec4b2faa-tmp-dir\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.170955 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.170006 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfbd346b-c505-44fb-828c-0fc38124c540-agent-certs\") pod \"konnectivity-agent-8xkv8\" (UID: \"cfbd346b-c505-44fb-828c-0fc38124c540\") " pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.173372 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.173342 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dda6c0d0-323a-4782-8f94-95dcb580be7e-etc-tuned\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.173532 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.173515 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:00:32.173608 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.173536 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:00:32.173608 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.173551 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:32.173714 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.173655 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:32.673634961 +0000 UTC m=+2.140609273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:32.176417 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.176383 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqqrn\" (UniqueName: \"kubernetes.io/projected/6196293c-156c-4f46-9f14-f228ec4b2faa-kube-api-access-cqqrn\") pod \"node-resolver-2npnt\" (UID: \"6196293c-156c-4f46-9f14-f228ec4b2faa\") " pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.176598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.176575 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj7rp\" (UniqueName: \"kubernetes.io/projected/5e017ac5-e1f6-4735-b095-7db2d6e18559-kube-api-access-zj7rp\") pod \"multus-nmxwn\" (UID: \"5e017ac5-e1f6-4735-b095-7db2d6e18559\") " pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.176839 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.176816 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x8bh\" (UniqueName: \"kubernetes.io/projected/dda6c0d0-323a-4782-8f94-95dcb580be7e-kube-api-access-7x8bh\") pod \"tuned-vjvvz\" (UID: \"dda6c0d0-323a-4782-8f94-95dcb580be7e\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.177491 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.177461 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-854gc\" (UniqueName: \"kubernetes.io/projected/c6d5c01e-3cbb-476f-b39c-9960a8e4beb7-kube-api-access-854gc\") pod \"node-ca-dz2bk\" (UID: \"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7\") " pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.177666 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.177639 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtv8q\" (UniqueName: \"kubernetes.io/projected/40039ba0-8699-446d-bbcb-270bed4a7beb-kube-api-access-vtv8q\") pod \"aws-ebs-csi-driver-node-rbms4\" (UID: \"40039ba0-8699-446d-bbcb-270bed4a7beb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.177776 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.177753 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f2n4\" (UniqueName: \"kubernetes.io/projected/94e5f207-84e3-4d24-8b1d-82bfd25e0c25-kube-api-access-8f2n4\") pod \"multus-additional-cni-plugins-vb4xn\" (UID: \"94e5f207-84e3-4d24-8b1d-82bfd25e0c25\") " pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.178080 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.178062 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwcvn\" (UniqueName: \"kubernetes.io/projected/ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0-kube-api-access-nwcvn\") pod \"iptables-alerter-767bx\" (UID: \"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0\") " pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.239706 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.239670 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631442f90a9aec34b0d279ea0f5f9c73.slice/crio-e2ae45c5ac0f46bf1e17865df33b821894694bb0afe6112433bac0e3d1ecdbc2 WatchSource:0}: Error finding container e2ae45c5ac0f46bf1e17865df33b821894694bb0afe6112433bac0e3d1ecdbc2: Status 404 returned error can't find the container with id e2ae45c5ac0f46bf1e17865df33b821894694bb0afe6112433bac0e3d1ecdbc2 Jul 15 19:00:32.245636 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.245620 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:00:32.294494 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.294464 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:32.305116 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.305094 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bdd365b_6d8b_4b34_b68f_0d17f9600606.slice/crio-7410e0eb2c323f5c5b616d042704a4b8e0b341613bf777c3cb70ebcc69a2b671 WatchSource:0}: Error finding container 7410e0eb2c323f5c5b616d042704a4b8e0b341613bf777c3cb70ebcc69a2b671: Status 404 returned error can't find the container with id 7410e0eb2c323f5c5b616d042704a4b8e0b341613bf777c3cb70ebcc69a2b671 Jul 15 19:00:32.312440 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.312421 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:32.319335 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.319313 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbd346b_c505_44fb_828c_0fc38124c540.slice/crio-121f4d9e74dab61071b18db12e5b773c5c96192c98191bdbc718524bab9493ef WatchSource:0}: Error finding container 121f4d9e74dab61071b18db12e5b773c5c96192c98191bdbc718524bab9493ef: Status 404 returned error can't find the container with id 121f4d9e74dab61071b18db12e5b773c5c96192c98191bdbc718524bab9493ef Jul 15 19:00:32.330597 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.330578 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" Jul 15 19:00:32.336818 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.336787 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" Jul 15 19:00:32.338038 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.338013 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40039ba0_8699_446d_bbcb_270bed4a7beb.slice/crio-c564dcdd1875b1d194a35daf860154bf560140bc5d3040696dbc99abde5d5c13 WatchSource:0}: Error finding container c564dcdd1875b1d194a35daf860154bf560140bc5d3040696dbc99abde5d5c13: Status 404 returned error can't find the container with id c564dcdd1875b1d194a35daf860154bf560140bc5d3040696dbc99abde5d5c13 Jul 15 19:00:32.344355 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.344333 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda6c0d0_323a_4782_8f94_95dcb580be7e.slice/crio-d77d9c49f20ca93b41fcd96b1a8bf7b04cef56b5f601dbf06da8dbde5971b2ae WatchSource:0}: Error finding container d77d9c49f20ca93b41fcd96b1a8bf7b04cef56b5f601dbf06da8dbde5971b2ae: Status 404 returned error can't find the container with id d77d9c49f20ca93b41fcd96b1a8bf7b04cef56b5f601dbf06da8dbde5971b2ae Jul 15 19:00:32.354753 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.354733 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2npnt" Jul 15 19:00:32.362050 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.362028 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6196293c_156c_4f46_9f14_f228ec4b2faa.slice/crio-0789cd14773537267a85bea62ad0cfda2add288e64641198644c226d7dc0cb56 WatchSource:0}: Error finding container 0789cd14773537267a85bea62ad0cfda2add288e64641198644c226d7dc0cb56: Status 404 returned error can't find the container with id 0789cd14773537267a85bea62ad0cfda2add288e64641198644c226d7dc0cb56 Jul 15 19:00:32.365267 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.365252 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" Jul 15 19:00:32.372470 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.372442 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e5f207_84e3_4d24_8b1d_82bfd25e0c25.slice/crio-edbccc835fa84affa92a1841755e94cca9a5dba77a3aa57d539d8c589a44b549 WatchSource:0}: Error finding container edbccc835fa84affa92a1841755e94cca9a5dba77a3aa57d539d8c589a44b549: Status 404 returned error can't find the container with id edbccc835fa84affa92a1841755e94cca9a5dba77a3aa57d539d8c589a44b549 Jul 15 19:00:32.383494 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.383471 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-767bx" Jul 15 19:00:32.391447 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.391427 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4a7f4a_559d_47f4_bb9a_98ab7bafffd0.slice/crio-7c1829fae18a246be875b7c97de118123156ddc8419c225256d601cb93c6e029 WatchSource:0}: Error finding container 7c1829fae18a246be875b7c97de118123156ddc8419c225256d601cb93c6e029: Status 404 returned error can't find the container with id 7c1829fae18a246be875b7c97de118123156ddc8419c225256d601cb93c6e029 Jul 15 19:00:32.391535 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.391467 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dz2bk" Jul 15 19:00:32.398668 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.398642 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d5c01e_3cbb_476f_b39c_9960a8e4beb7.slice/crio-4dcdecca8824775dabdbc9ba11977fa5fad0f3bee8aa55646499a798b7e42b60 WatchSource:0}: Error finding container 4dcdecca8824775dabdbc9ba11977fa5fad0f3bee8aa55646499a798b7e42b60: Status 404 returned error can't find the container with id 4dcdecca8824775dabdbc9ba11977fa5fad0f3bee8aa55646499a798b7e42b60 Jul 15 19:00:32.408400 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.408379 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nmxwn" Jul 15 19:00:32.415370 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:00:32.415349 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e017ac5_e1f6_4735_b095_7db2d6e18559.slice/crio-7d5b76c1316ee8b1f889f17af63a8e749560ddc9ce9c266b592e707edc265b73 WatchSource:0}: Error finding container 7d5b76c1316ee8b1f889f17af63a8e749560ddc9ce9c266b592e707edc265b73: Status 404 returned error can't find the container with id 7d5b76c1316ee8b1f889f17af63a8e749560ddc9ce9c266b592e707edc265b73 Jul 15 19:00:32.527253 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.527170 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:00:32.571596 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.571558 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:32.571756 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.571680 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:32.571756 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.571739 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:33.571724832 +0000 UTC m=+3.038699123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:32.671949 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.671911 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:32.672119 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.672061 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:32.672162 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.672128 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:33.672112716 +0000 UTC m=+3.139087007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:32.773022 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:32.772977 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:32.773225 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.773163 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:00:32.773225 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.773181 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:00:32.773225 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.773190 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:32.773432 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:32.773253 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:33.773231914 +0000 UTC m=+3.240206206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:33.002613 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.002484 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 18:55:32 +0000 UTC" deadline="2028-03-02 20:46:05.355568722 +0000 UTC" Jul 15 19:00:33.002613 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.002551 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14305h45m32.353022097s" Jul 15 19:00:33.027411 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.027380 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:33.027593 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.027528 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:33.035258 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.035204 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"7410e0eb2c323f5c5b616d042704a4b8e0b341613bf777c3cb70ebcc69a2b671"} Jul 15 19:00:33.039784 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.039754 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" event={"ID":"631442f90a9aec34b0d279ea0f5f9c73","Type":"ContainerStarted","Data":"e2ae45c5ac0f46bf1e17865df33b821894694bb0afe6112433bac0e3d1ecdbc2"} Jul 15 19:00:33.044630 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.044564 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dz2bk" event={"ID":"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7","Type":"ContainerStarted","Data":"4dcdecca8824775dabdbc9ba11977fa5fad0f3bee8aa55646499a798b7e42b60"} Jul 15 19:00:33.047073 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.047049 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2npnt" event={"ID":"6196293c-156c-4f46-9f14-f228ec4b2faa","Type":"ContainerStarted","Data":"0789cd14773537267a85bea62ad0cfda2add288e64641198644c226d7dc0cb56"} Jul 15 19:00:33.051154 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.051125 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" event={"ID":"dda6c0d0-323a-4782-8f94-95dcb580be7e","Type":"ContainerStarted","Data":"d77d9c49f20ca93b41fcd96b1a8bf7b04cef56b5f601dbf06da8dbde5971b2ae"} Jul 15 19:00:33.057775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.057736 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8xkv8" event={"ID":"cfbd346b-c505-44fb-828c-0fc38124c540","Type":"ContainerStarted","Data":"121f4d9e74dab61071b18db12e5b773c5c96192c98191bdbc718524bab9493ef"} Jul 15 19:00:33.061031 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.061005 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nmxwn" event={"ID":"5e017ac5-e1f6-4735-b095-7db2d6e18559","Type":"ContainerStarted","Data":"7d5b76c1316ee8b1f889f17af63a8e749560ddc9ce9c266b592e707edc265b73"} Jul 15 19:00:33.063162 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.063138 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-767bx" event={"ID":"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0","Type":"ContainerStarted","Data":"7c1829fae18a246be875b7c97de118123156ddc8419c225256d601cb93c6e029"} Jul 15 19:00:33.067206 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.067178 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerStarted","Data":"edbccc835fa84affa92a1841755e94cca9a5dba77a3aa57d539d8c589a44b549"} Jul 15 19:00:33.070649 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.070622 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" event={"ID":"40039ba0-8699-446d-bbcb-270bed4a7beb","Type":"ContainerStarted","Data":"c564dcdd1875b1d194a35daf860154bf560140bc5d3040696dbc99abde5d5c13"} Jul 15 19:00:33.579206 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.579163 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:33.587586 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.587553 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:33.587752 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.587651 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:35.587628307 +0000 UTC m=+5.054602613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:33.680799 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.680757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:33.681003 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.680938 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:33.681078 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.681022 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:35.681002047 +0000 UTC m=+5.147976343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:33.781978 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:33.781941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:33.782342 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.782150 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:00:33.782342 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.782172 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:00:33.782342 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.782186 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:33.782342 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:33.782246 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:35.782225909 +0000 UTC m=+5.249200207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:34.003699 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:34.003580 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 18:55:32 +0000 UTC" deadline="2028-01-10 23:23:10.5750845 +0000 UTC" Jul 15 19:00:34.003699 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:34.003647 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13060h22m36.571442318s" Jul 15 19:00:34.028149 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:34.028119 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:34.028346 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:34.028256 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:34.028714 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:34.028696 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:34.028831 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:34.028810 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:34.095544 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:34.095503 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerStarted","Data":"cf8c63d9af8732e21db07318dfc27457e15b4f802b7cb572200baf3f1f00fb41"} Jul 15 19:00:34.317200 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:34.317116 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:00:35.030842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:35.028852 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:35.030842 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.029016 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:35.598278 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:35.597572 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:35.598278 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.597772 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:35.598278 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.597840 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:39.597818511 +0000 UTC m=+9.064792806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:35.698021 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:35.697978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:35.698228 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.698142 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:35.698228 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.698206 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:39.69818827 +0000 UTC m=+9.165162576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:35.799121 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:35.799022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:35.799361 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.799210 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:00:35.799361 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.799234 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:00:35.799361 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.799263 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:35.799361 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:35.799328 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:39.799309144 +0000 UTC m=+9.266283451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:36.027816 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:36.027778 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:36.028001 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:36.027941 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:36.028533 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:36.028339 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:36.028533 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:36.028465 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:37.027835 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:37.027793 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:37.028375 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:37.027947 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:38.027860 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:38.027804 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:38.028398 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:38.027993 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:38.028398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:38.028078 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:38.028398 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:38.028195 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:39.032397 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:39.032362 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:39.032895 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.032500 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:39.637688 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:39.637646 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:39.637881 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.637864 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:39.637962 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.637938 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:47.63791693 +0000 UTC m=+17.104891230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:39.738949 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:39.738895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:39.739155 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.739089 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:39.739223 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.739156 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:47.739134738 +0000 UTC m=+17.206109046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:39.840357 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:39.840244 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:39.840520 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.840416 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:00:39.840520 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.840437 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:00:39.840520 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.840450 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:39.840520 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:39.840518 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:00:47.840498303 +0000 UTC m=+17.307472618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:40.028103 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:40.028070 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:40.028332 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:40.028070 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:40.028332 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:40.028239 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:40.028464 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:40.028352 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:41.029443 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:41.029406 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:41.029945 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:41.029559 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:42.027415 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:42.027371 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:42.027619 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:42.027534 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:42.028068 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:42.027917 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:42.028068 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:42.028020 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:43.030855 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:43.030821 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:43.031289 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:43.030937 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:44.027857 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:44.027821 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:44.028036 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:44.027832 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:44.028036 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:44.027957 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:44.028194 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:44.028035 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:45.027387 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:45.027352 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:45.027849 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:45.027481 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:46.027360 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:46.027327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:46.027360 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:46.027346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:46.027890 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:46.027466 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:46.027890 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:46.027596 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:47.028094 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:47.028056 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:47.028537 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.028189 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:47.701200 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:47.701154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:47.701402 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.701320 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:47.701471 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.701405 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:03.701382372 +0000 UTC m=+33.168356668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:00:47.801826 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:47.801788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:47.801989 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.801952 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:47.802050 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.802028 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:03.802007119 +0000 UTC m=+33.268981412 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:00:47.902524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:47.902483 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:47.902747 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.902636 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:00:47.902747 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.902655 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:00:47.902747 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.902668 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:47.902747 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:47.902728 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:03.902711885 +0000 UTC m=+33.369686192 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:00:48.027618 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:48.027404 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:48.027839 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:48.027420 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:48.027839 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:48.027679 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:48.027839 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:48.027777 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:49.028600 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:49.028132 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:49.028600 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:49.028268 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:50.028172 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:50.028133 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:50.028351 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:50.028133 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:50.028351 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:50.028275 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:50.028351 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:50.028341 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:51.030190 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:51.030161 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:51.030597 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:51.030273 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:52.027837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:52.027804 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:52.027998 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:52.027929 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:52.027998 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:52.027983 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:52.028134 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:52.028114 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:53.027520 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:53.027484 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:53.027981 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:53.027617 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:54.028033 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.028007 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:54.028463 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.028007 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:54.028463 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:54.028152 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:54.028463 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:54.028182 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:54.138981 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.138921 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" event={"ID":"dda6c0d0-323a-4782-8f94-95dcb580be7e","Type":"ContainerStarted","Data":"c0db92c89a0607364c5286b77e3a6e471fed532b47896123da6c389f72c67327"} Jul 15 19:00:54.140808 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.140744 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8xkv8" event={"ID":"cfbd346b-c505-44fb-828c-0fc38124c540","Type":"ContainerStarted","Data":"4a14b3d917088ca1a44a83dc0fb31cd33a79e15939edd2807b61783542c0585d"} Jul 15 19:00:54.143371 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.143260 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerStarted","Data":"1ceeb6d74d6726a15c6e9ecfda3267a5775868e37fab17f87143c2c177afa168"} Jul 15 19:00:54.146306 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.146268 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nmxwn" event={"ID":"5e017ac5-e1f6-4735-b095-7db2d6e18559","Type":"ContainerStarted","Data":"5adbbd47d01ac637ceaf1e42c3bdaebc515edecf88969a0ef950a1589893b1cc"} Jul 15 19:00:54.147966 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.147937 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerStarted","Data":"f8e38c14df05483e99f0b7b5311aa06af4e6e8fa917e83d7db515bcf6cfb1c1c"} Jul 15 19:00:54.149387 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.149325 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" event={"ID":"40039ba0-8699-446d-bbcb-270bed4a7beb","Type":"ContainerStarted","Data":"4fb11feda9646530166e478671ff09f423850f1b9b3fb466246a272d66d3c336"} Jul 15 19:00:54.151179 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.151160 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:00:54.151572 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.151543 2572 generic.go:356] "Generic (PLEG): container finished" podID="7bdd365b-6d8b-4b34-b68f-0d17f9600606" containerID="5b8d49e6448c0557dfb8b708c8238baea7271928f99b13c9a18949e71a82d8df" exitCode=1 Jul 15 19:00:54.151663 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.151604 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerDied","Data":"5b8d49e6448c0557dfb8b708c8238baea7271928f99b13c9a18949e71a82d8df"} Jul 15 19:00:54.151731 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.151716 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"1f38a810f73a084b2c09af5034b57ada32daeb8798de05c12fb08ce05cae6444"} Jul 15 19:00:54.155806 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.155763 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" event={"ID":"631442f90a9aec34b0d279ea0f5f9c73","Type":"ContainerStarted","Data":"b9418e5ef1f28b2790f59a3beb198be3ce6d71b897ccc50d978c2b0f04cd0556"} Jul 15 19:00:54.160010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.159970 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vjvvz" podStartSLOduration=0.976683203 podStartE2EDuration="22.15995186s" totalImagesPullingTime="21.183268657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:54.15936775 +0000 UTC m=+23.626342063" watchObservedRunningTime="2026-07-15 19:00:54.15995186 +0000 UTC m=+23.626926175" Jul 15 19:00:54.160543 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.160228 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dz2bk" event={"ID":"c6d5c01e-3cbb-476f-b39c-9960a8e4beb7","Type":"ContainerStarted","Data":"1ac3e026b3019228f5fff61a70877170e4e402ad62023442b716bc285f341a01"} Jul 15 19:00:54.161959 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.161930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2npnt" event={"ID":"6196293c-156c-4f46-9f14-f228ec4b2faa","Type":"ContainerStarted","Data":"9c5b23bca0975d8781363262d2f501b722c51772167ea81c02b55cea23baeeba"} Jul 15 19:00:54.176893 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.176857 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8xkv8" podStartSLOduration=0.999917426 podStartE2EDuration="22.176844519s" totalImagesPullingTime="21.176927093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:54.17628422 +0000 UTC m=+23.643258532" watchObservedRunningTime="2026-07-15 19:00:54.176844519 +0000 UTC m=+23.643818828" Jul 15 19:00:54.232088 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.232048 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nmxwn" podStartSLOduration=1.117908223 podStartE2EDuration="22.232033701s" totalImagesPullingTime="21.114125478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:54.232027855 +0000 UTC m=+23.699002177" watchObservedRunningTime="2026-07-15 19:00:54.232033701 +0000 UTC m=+23.699007994" Jul 15 19:00:54.247823 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.247771 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" podStartSLOduration=22.247753757 podStartE2EDuration="22.247753757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:54.247526156 +0000 UTC m=+23.714500467" watchObservedRunningTime="2026-07-15 19:00:54.247753757 +0000 UTC m=+23.714728069" Jul 15 19:00:54.284971 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.284850 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dz2bk" podStartSLOduration=1.186971322 podStartE2EDuration="22.284835621s" totalImagesPullingTime="21.097864299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:54.266574388 +0000 UTC m=+23.733548699" watchObservedRunningTime="2026-07-15 19:00:54.284835621 +0000 UTC m=+23.751809961" Jul 15 19:00:54.285448 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.285423 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2npnt" podStartSLOduration=5.458044459 podStartE2EDuration="22.285414697s" totalImagesPullingTime="16.827370238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:54.285103137 +0000 UTC m=+23.752077448" watchObservedRunningTime="2026-07-15 19:00:54.285414697 +0000 UTC m=+23.752389008" Jul 15 19:00:54.392365 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.392325 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:54.393200 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:54.393179 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:55.031954 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.031922 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:55.032647 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:55.032035 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:55.165155 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.165120 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-767bx" event={"ID":"ee4a7f4a-559d-47f4-bb9a-98ab7bafffd0","Type":"ContainerStarted","Data":"517db0659ec2128d67d0127f7654273e74e74dbb8582c1534993e70ac5a3d02e"} Jul 15 19:00:55.166476 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.166447 2572 generic.go:356] "Generic (PLEG): container finished" podID="94e5f207-84e3-4d24-8b1d-82bfd25e0c25" containerID="f8e38c14df05483e99f0b7b5311aa06af4e6e8fa917e83d7db515bcf6cfb1c1c" exitCode=0 Jul 15 19:00:55.166592 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.166527 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerDied","Data":"f8e38c14df05483e99f0b7b5311aa06af4e6e8fa917e83d7db515bcf6cfb1c1c"} Jul 15 19:00:55.169369 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.169353 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:00:55.169781 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.169762 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"7fcb7da3e3987c0c4b8da79dbbe978c29107ef3d9c52f2d18a3bab49dc15b850"} Jul 15 19:00:55.169879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.169832 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"99e0d59a94f72aea5b5879b69381da8e8bbd5c1e47594cb328a233b147d3a9a1"} Jul 15 19:00:55.169942 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.169888 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"7134f41cfdb27d396849bebd0c502fae05015f5b8a1b93405ea00e40c10077ae"} Jul 15 19:00:55.169942 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.169938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"108783d2d83a3f05bd0e736c51ba8eb9fee7c79595480daa6fe20eaf82ccf9a7"} Jul 15 19:00:55.171158 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.171138 2572 generic.go:356] "Generic (PLEG): container finished" podID="6fca482dcad147ae6c62a690d3cba064" containerID="1ceeb6d74d6726a15c6e9ecfda3267a5775868e37fab17f87143c2c177afa168" exitCode=0 Jul 15 19:00:55.171282 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.171260 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerDied","Data":"1ceeb6d74d6726a15c6e9ecfda3267a5775868e37fab17f87143c2c177afa168"} Jul 15 19:00:55.171477 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.171464 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:55.171931 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.171915 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8xkv8" Jul 15 19:00:55.180335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.180299 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-767bx" podStartSLOduration=2.040660023 podStartE2EDuration="23.180287015s" totalImagesPullingTime="21.139626992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:55.180009082 +0000 UTC m=+24.646983393" watchObservedRunningTime="2026-07-15 19:00:55.180287015 +0000 UTC m=+24.647261307" Jul 15 19:00:55.567197 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:55.567175 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 19:00:56.027879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.027842 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:56.028304 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.027842 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:56.028304 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:56.027967 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:56.028304 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:56.028042 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:56.164762 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.164676 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:00:55.56719116Z","UUID":"8578d810-c2ad-4da5-9dfa-e5fe6be1c47d","Handler":null,"Name":"","Endpoint":""} Jul 15 19:00:56.169293 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.169273 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 19:00:56.169419 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.169300 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 19:00:56.175199 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.175175 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerStarted","Data":"d42a86ecbe36eee895e60afc0180c315a5d9712cd24876fa8100426b31e05426"} Jul 15 19:00:56.177006 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.176983 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" event={"ID":"40039ba0-8699-446d-bbcb-270bed4a7beb","Type":"ContainerStarted","Data":"ab51b2b4284c7e39c9d7ca47f4bb565584f69ccf97299fd1cc9fbd9612d96f8f"} Jul 15 19:00:56.191899 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:56.191863 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" podStartSLOduration=24.19185129 podStartE2EDuration="24.19185129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:56.191408293 +0000 UTC m=+25.658382603" watchObservedRunningTime="2026-07-15 19:00:56.19185129 +0000 UTC m=+25.658825601" Jul 15 19:00:57.027631 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:57.027560 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:57.027793 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:57.027676 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:57.181904 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:57.181876 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:00:57.182318 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:57.182262 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"9347ac604965ca5c1acac7343d1f7888834080b7fead554509c2e72c7eb64af4"} Jul 15 19:00:58.028177 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:58.028141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:00:58.028407 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:58.028141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:00:58.028407 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:58.028280 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:00:58.028407 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:58.028342 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:00:59.028527 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.028320 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:00:59.029292 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:00:59.028662 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:00:59.191218 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.191194 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:00:59.191598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.191572 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"61e33c5ad7f6f7f8a4c35d87fe290424bf9c3b90be7855826c40ef491e411e3d"} Jul 15 19:00:59.191894 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.191866 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:59.192005 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.191951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:59.192005 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.192002 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:59.192169 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.192153 2572 scope.go:117] "RemoveContainer" containerID="5b8d49e6448c0557dfb8b708c8238baea7271928f99b13c9a18949e71a82d8df" Jul 15 19:00:59.212431 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.212389 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:00:59.215649 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:00:59.215626 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:01:00.027922 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:00.027891 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:00.028105 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:00.027900 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:00.028105 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:00.028007 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:01:00.028105 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:00.028095 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:01:00.199995 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:00.199970 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:01:00.200437 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:00.200412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" event={"ID":"7bdd365b-6d8b-4b34-b68f-0d17f9600606","Type":"ContainerStarted","Data":"928d22695701f89b9ab85be1e01630048e21077ddca0c771f61cda61aa92cf88"} Jul 15 19:01:01.029415 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.029383 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:01.029609 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:01.029499 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:01:01.191547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.190968 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" podStartSLOduration=7.702628907 podStartE2EDuration="29.190947862s" totalImagesPullingTime="21.488318955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:00.231128751 +0000 UTC m=+29.698103063" watchObservedRunningTime="2026-07-15 19:01:01.190947862 +0000 UTC m=+30.657922173" Jul 15 19:01:01.192326 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.192301 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4dg5"] Jul 15 19:01:01.192464 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.192444 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:01.192570 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:01.192548 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:01:01.194003 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.193979 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kt4sp"] Jul 15 19:01:01.194138 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.194093 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:01.194217 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:01.194193 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:01:01.207902 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.207877 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bmgw"] Jul 15 19:01:01.208300 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:01.207988 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:01.208300 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:01.208076 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:01:03.027526 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.027502 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:03.027909 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.027504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:03.027909 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.027645 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:01:03.027909 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.027503 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:03.027909 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.027698 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:01:03.027909 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.027740 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:01:03.210282 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.210046 2572 generic.go:356] "Generic (PLEG): container finished" podID="94e5f207-84e3-4d24-8b1d-82bfd25e0c25" containerID="e168732688f672587bc0caa1e5cd3a448d6a44e5621f5ab8a2131fe601f98b64" exitCode=0 Jul 15 19:01:03.210282 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.210129 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerDied","Data":"e168732688f672587bc0caa1e5cd3a448d6a44e5621f5ab8a2131fe601f98b64"} Jul 15 19:01:03.212427 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.212404 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" event={"ID":"40039ba0-8699-446d-bbcb-270bed4a7beb","Type":"ContainerStarted","Data":"cf1d21d56e7a76e93db928e02ba25107368684184582551e5ad6aca55ae65c34"} Jul 15 19:01:03.253252 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.253181 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rbms4" podStartSLOduration=1.406541786 podStartE2EDuration="31.253162785s" totalImagesPullingTime="29.846620999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:03.252955064 +0000 UTC m=+32.719929376" watchObservedRunningTime="2026-07-15 19:01:03.253162785 +0000 UTC m=+32.720137097" Jul 15 19:01:03.720060 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.720027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:03.720256 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.720198 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:01:03.720327 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.720286 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:35.72027104 +0000 UTC m=+65.187245328 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:01:03.820470 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.820441 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:03.820656 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.820606 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:01:03.820719 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.820687 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret podName:35b4663c-ea09-4be7-a31c-660e76c46799 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:35.82066137 +0000 UTC m=+65.287635675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret") pod "global-pull-secret-syncer-b4dg5" (UID: "35b4663c-ea09-4be7-a31c-660e76c46799") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:01:03.920820 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:03.920784 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:03.921019 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.920997 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:01:03.921109 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.921033 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:01:03.921109 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.921047 2572 projected.go:196] Error preparing data for projected volume kube-api-access-rh46v for pod openshift-network-diagnostics/network-check-target-2bmgw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:01:03.921208 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:03.921117 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v podName:a1c74c7b-4982-4e82-a1a6-4f2819ac9f36 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:35.921097492 +0000 UTC m=+65.388071796 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rh46v" (UniqueName: "kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v") pod "network-check-target-2bmgw" (UID: "a1c74c7b-4982-4e82-a1a6-4f2819ac9f36") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:01:05.027670 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:05.027639 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:05.028307 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:05.027639 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:05.028307 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:05.027760 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4sp" podUID="8e8c81ac-0a8b-4100-8e3b-d0472538bcb5" Jul 15 19:01:05.028307 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:05.027770 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:05.028307 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:05.027874 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bmgw" podUID="a1c74c7b-4982-4e82-a1a6-4f2819ac9f36" Jul 15 19:01:05.028307 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:05.027951 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4dg5" podUID="35b4663c-ea09-4be7-a31c-660e76c46799" Jul 15 19:01:05.217594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:05.217565 2572 generic.go:356] "Generic (PLEG): container finished" podID="94e5f207-84e3-4d24-8b1d-82bfd25e0c25" containerID="570b318e7d1092a5405d1f6f0e0c50764e6eb6518fde0cdc2742e84b3075acfc" exitCode=0 Jul 15 19:01:05.217745 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:05.217615 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerDied","Data":"570b318e7d1092a5405d1f6f0e0c50764e6eb6518fde0cdc2742e84b3075acfc"} Jul 15 19:01:06.342497 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.342470 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeReady" Jul 15 19:01:06.342866 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.342580 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 19:01:06.384417 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.384388 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b5d67c5b9-sjnnt"] Jul 15 19:01:06.386064 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.386049 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.388692 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.388672 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 19:01:06.388810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.388697 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 19:01:06.388966 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.388952 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 19:01:06.389061 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.389042 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z9lvq\"" Jul 15 19:01:06.398392 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.397137 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z65bn"] Jul 15 19:01:06.398803 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.398786 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 19:01:06.399440 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.399426 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.402442 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.402421 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dgrg9\"" Jul 15 19:01:06.402542 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.402474 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 19:01:06.403181 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.403150 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 19:01:06.403819 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.403801 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c79mn"] Jul 15 19:01:06.405550 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.405462 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5d67c5b9-sjnnt"] Jul 15 19:01:06.405550 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.405536 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.408795 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.408775 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 19:01:06.408868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.408809 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 19:01:06.408868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.408819 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jgfdg\"" Jul 15 19:01:06.409314 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.409298 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 19:01:06.413546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.413526 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z65bn"] Jul 15 19:01:06.426556 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.426530 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c79mn"] Jul 15 19:01:06.538737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538659 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de406e7-cfc2-4a52-b55b-c7819d750e2a-tmp-dir\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.538737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538691 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwxfn\" (UniqueName: \"kubernetes.io/projected/9de406e7-cfc2-4a52-b55b-c7819d750e2a-kube-api-access-rwxfn\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.538737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538709 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-image-registry-private-configuration\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.538737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538725 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.538952 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538764 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba7f537-3e9b-41bc-ba46-cf83477ec8a4-cert\") pod \"ingress-canary-c79mn\" (UID: \"dba7f537-3e9b-41bc-ba46-cf83477ec8a4\") " pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.538952 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538861 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/361939d2-4fef-45ba-9488-1b42cc04c7a9-ca-trust-extracted\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.538952 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538885 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w7tb\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-kube-api-access-4w7tb\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.538952 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538914 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-trusted-ca\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.538952 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q88rb\" (UniqueName: \"kubernetes.io/projected/dba7f537-3e9b-41bc-ba46-cf83477ec8a4-kube-api-access-q88rb\") pod \"ingress-canary-c79mn\" (UID: \"dba7f537-3e9b-41bc-ba46-cf83477ec8a4\") " pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.539095 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.538982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.539095 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.539001 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-bound-sa-token\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.539095 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.539034 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-installation-pull-secrets\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.539095 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.539087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9de406e7-cfc2-4a52-b55b-c7819d750e2a-config-volume\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.539209 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.539106 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-certificates\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.554631 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.554597 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xkv8_cfbd346b-c505-44fb-828c-0fc38124c540/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:06.639707 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.639707 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639713 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-bound-sa-token\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.639863 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-installation-pull-secrets\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.639863 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9de406e7-cfc2-4a52-b55b-c7819d750e2a-config-volume\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.639863 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639825 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-certificates\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.639863 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de406e7-cfc2-4a52-b55b-c7819d750e2a-tmp-dir\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwxfn\" (UniqueName: \"kubernetes.io/projected/9de406e7-cfc2-4a52-b55b-c7819d750e2a-kube-api-access-rwxfn\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639906 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-image-registry-private-configuration\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639927 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639950 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba7f537-3e9b-41bc-ba46-cf83477ec8a4-cert\") pod \"ingress-canary-c79mn\" (UID: \"dba7f537-3e9b-41bc-ba46-cf83477ec8a4\") " pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.639981 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/361939d2-4fef-45ba-9488-1b42cc04c7a9-ca-trust-extracted\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640005 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w7tb\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-kube-api-access-4w7tb\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:06.640018 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640040 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-trusted-ca\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:06.640078 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls podName:9de406e7-cfc2-4a52-b55b-c7819d750e2a nodeName:}" failed. No retries permitted until 2026-07-15 19:01:07.140058502 +0000 UTC m=+36.607032804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls") pod "dns-default-z65bn" (UID: "9de406e7-cfc2-4a52-b55b-c7819d750e2a") : secret "dns-default-metrics-tls" not found Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:06.640078 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:01:06.640117 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:06.640097 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5d67c5b9-sjnnt: secret "image-registry-tls" not found Jul 15 19:01:06.640655 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:06.640132 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls podName:361939d2-4fef-45ba-9488-1b42cc04c7a9 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:07.140122477 +0000 UTC m=+36.607096799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls") pod "image-registry-5b5d67c5b9-sjnnt" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9") : secret "image-registry-tls" not found Jul 15 19:01:06.640655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640278 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de406e7-cfc2-4a52-b55b-c7819d750e2a-tmp-dir\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.640655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640330 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q88rb\" (UniqueName: \"kubernetes.io/projected/dba7f537-3e9b-41bc-ba46-cf83477ec8a4-kube-api-access-q88rb\") pod \"ingress-canary-c79mn\" (UID: \"dba7f537-3e9b-41bc-ba46-cf83477ec8a4\") " pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.640655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640437 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9de406e7-cfc2-4a52-b55b-c7819d750e2a-config-volume\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.640655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640498 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/361939d2-4fef-45ba-9488-1b42cc04c7a9-ca-trust-extracted\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.640834 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.640763 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-certificates\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.641280 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.641259 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-trusted-ca\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.644219 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.644195 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-image-registry-private-configuration\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.644330 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.644202 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-installation-pull-secrets\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.644375 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.644346 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba7f537-3e9b-41bc-ba46-cf83477ec8a4-cert\") pod \"ingress-canary-c79mn\" (UID: \"dba7f537-3e9b-41bc-ba46-cf83477ec8a4\") " pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.646122 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.646097 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xkv8_cfbd346b-c505-44fb-828c-0fc38124c540/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:06.649469 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.649445 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w7tb\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-kube-api-access-4w7tb\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.649917 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.649893 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwxfn\" (UniqueName: \"kubernetes.io/projected/9de406e7-cfc2-4a52-b55b-c7819d750e2a-kube-api-access-rwxfn\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:06.649997 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.649944 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-bound-sa-token\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:06.650082 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.650065 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q88rb\" (UniqueName: \"kubernetes.io/projected/dba7f537-3e9b-41bc-ba46-cf83477ec8a4-kube-api-access-q88rb\") pod \"ingress-canary-c79mn\" (UID: \"dba7f537-3e9b-41bc-ba46-cf83477ec8a4\") " pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.716076 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.716045 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c79mn" Jul 15 19:01:06.845396 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:06.845208 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c79mn"] Jul 15 19:01:06.848179 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:06.848159 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba7f537_3e9b_41bc_ba46_cf83477ec8a4.slice/crio-8bdddcfd15578c730bcec34957141bd3364d0d6aaeeefd26905eaac550b3a4a7 WatchSource:0}: Error finding container 8bdddcfd15578c730bcec34957141bd3364d0d6aaeeefd26905eaac550b3a4a7: Status 404 returned error can't find the container with id 8bdddcfd15578c730bcec34957141bd3364d0d6aaeeefd26905eaac550b3a4a7 Jul 15 19:01:07.027420 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.027382 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:07.027555 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.027391 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:07.027633 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.027391 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:07.030663 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.030640 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:01:07.030663 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.030656 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7hbwd\"" Jul 15 19:01:07.030842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.030706 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:01:07.030842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.030752 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:01:07.030842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.030659 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:01:07.030842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.030785 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j7rjq\"" Jul 15 19:01:07.144006 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.143975 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:07.144181 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.144037 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:07.144181 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:07.144115 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:01:07.144181 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:07.144174 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:01:07.144390 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:07.144192 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5d67c5b9-sjnnt: secret "image-registry-tls" not found Jul 15 19:01:07.144390 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:07.144178 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls podName:9de406e7-cfc2-4a52-b55b-c7819d750e2a nodeName:}" failed. No retries permitted until 2026-07-15 19:01:08.144163387 +0000 UTC m=+37.611137675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls") pod "dns-default-z65bn" (UID: "9de406e7-cfc2-4a52-b55b-c7819d750e2a") : secret "dns-default-metrics-tls" not found Jul 15 19:01:07.144390 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:07.144273 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls podName:361939d2-4fef-45ba-9488-1b42cc04c7a9 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:08.144256999 +0000 UTC m=+37.611231288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls") pod "image-registry-5b5d67c5b9-sjnnt" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9") : secret "image-registry-tls" not found Jul 15 19:01:07.223179 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:07.223149 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c79mn" event={"ID":"dba7f537-3e9b-41bc-ba46-cf83477ec8a4","Type":"ContainerStarted","Data":"8bdddcfd15578c730bcec34957141bd3364d0d6aaeeefd26905eaac550b3a4a7"} Jul 15 19:01:08.153525 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:08.153489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:08.154011 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:08.153555 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:08.154011 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:08.153646 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:01:08.154011 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:08.153713 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls podName:9de406e7-cfc2-4a52-b55b-c7819d750e2a nodeName:}" failed. No retries permitted until 2026-07-15 19:01:10.153698436 +0000 UTC m=+39.620672724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls") pod "dns-default-z65bn" (UID: "9de406e7-cfc2-4a52-b55b-c7819d750e2a") : secret "dns-default-metrics-tls" not found Jul 15 19:01:08.154011 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:08.153651 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:01:08.154011 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:08.153749 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5d67c5b9-sjnnt: secret "image-registry-tls" not found Jul 15 19:01:08.154011 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:08.153791 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls podName:361939d2-4fef-45ba-9488-1b42cc04c7a9 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:10.153780316 +0000 UTC m=+39.620754604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls") pod "image-registry-5b5d67c5b9-sjnnt" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9") : secret "image-registry-tls" not found Jul 15 19:01:08.230417 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:08.230385 2572 generic.go:356] "Generic (PLEG): container finished" podID="94e5f207-84e3-4d24-8b1d-82bfd25e0c25" containerID="0edc7137391af8d9999e786ae876cf1388aaf0e39d7c485b73ce9b40156bdc1c" exitCode=0 Jul 15 19:01:08.230566 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:08.230435 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerDied","Data":"0edc7137391af8d9999e786ae876cf1388aaf0e39d7c485b73ce9b40156bdc1c"} Jul 15 19:01:09.974374 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:09.974343 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xkv8_cfbd346b-c505-44fb-828c-0fc38124c540/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:10.078325 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:10.078109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8xkv8_cfbd346b-c505-44fb-828c-0fc38124c540/konnectivity-agent/0.log" limit=1024 Jul 15 19:01:10.171294 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:10.171194 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:10.171435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:10.171299 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:10.171435 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:10.171368 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:01:10.171435 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:10.171422 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:01:10.171435 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:10.171435 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5d67c5b9-sjnnt: secret "image-registry-tls" not found Jul 15 19:01:10.171587 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:10.171442 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls podName:9de406e7-cfc2-4a52-b55b-c7819d750e2a nodeName:}" failed. No retries permitted until 2026-07-15 19:01:14.171420776 +0000 UTC m=+43.638395070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls") pod "dns-default-z65bn" (UID: "9de406e7-cfc2-4a52-b55b-c7819d750e2a") : secret "dns-default-metrics-tls" not found Jul 15 19:01:10.171587 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:10.171476 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls podName:361939d2-4fef-45ba-9488-1b42cc04c7a9 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:14.171462505 +0000 UTC m=+43.638436828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls") pod "image-registry-5b5d67c5b9-sjnnt" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9") : secret "image-registry-tls" not found Jul 15 19:01:10.238617 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:10.238583 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c79mn" event={"ID":"dba7f537-3e9b-41bc-ba46-cf83477ec8a4","Type":"ContainerStarted","Data":"3d2c1a3a15e4c1cce068a9e1a383a4f4b6563d6f91cdaf473995928294991876"} Jul 15 19:01:10.260469 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:10.260414 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c79mn" podStartSLOduration=1.201155831 podStartE2EDuration="4.260384417s" totalImagesPullingTime="3.059228586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:10.259225119 +0000 UTC m=+39.726199432" watchObservedRunningTime="2026-07-15 19:01:10.260384417 +0000 UTC m=+39.727358729" Jul 15 19:01:11.491037 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.491005 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr"] Jul 15 19:01:11.518567 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.518536 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr"] Jul 15 19:01:11.518730 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.518639 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.521897 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.521870 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 19:01:11.522020 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.521960 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 19:01:11.523041 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.523020 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 19:01:11.523147 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.523064 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4fh54\"" Jul 15 19:01:11.524738 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.524717 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 19:01:11.599888 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.599850 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf"] Jul 15 19:01:11.618766 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.618731 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf"] Jul 15 19:01:11.618927 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.618870 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" Jul 15 19:01:11.627954 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.627890 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gcvq6\"" Jul 15 19:01:11.627954 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.627894 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:01:11.633206 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.633180 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 19:01:11.635726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.635703 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-2kz84"] Jul 15 19:01:11.635857 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.635840 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:11.642213 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.642001 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-699qz\"" Jul 15 19:01:11.642531 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.642510 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 19:01:11.642765 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.642516 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:01:11.643362 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.643333 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 19:01:11.653709 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.653671 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6dffcc884d-qt2hp"] Jul 15 19:01:11.654723 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.654398 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.658413 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.658392 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 19:01:11.658516 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.658413 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:01:11.658516 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.658441 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bsl7h\"" Jul 15 19:01:11.658516 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.658470 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 19:01:11.658724 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.658709 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 19:01:11.667689 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.667668 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 19:01:11.680035 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.680013 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf"] Jul 15 19:01:11.680135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.680042 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dffcc884d-qt2hp"] Jul 15 19:01:11.680135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.680059 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf"] Jul 15 19:01:11.680135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.680083 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-2kz84"] Jul 15 19:01:11.680320 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.680163 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.682964 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.682927 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 19:01:11.683071 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.683036 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 19:01:11.683740 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.683722 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xxz89\"" Jul 15 19:01:11.683845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.683746 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 19:01:11.683845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.683722 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 19:01:11.683845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.683729 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 19:01:11.683845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.683724 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 19:01:11.684568 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.684550 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/613706c9-971d-4afb-bec1-093b211a3b51-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.684662 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.684617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.684720 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.684683 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwts7\" (UniqueName: \"kubernetes.io/projected/613706c9-971d-4afb-bec1-093b211a3b51-kube-api-access-rwts7\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.699078 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.699053 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hlmwm"] Jul 15 19:01:11.716036 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.716013 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.720835 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.720811 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 19:01:11.721331 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.721310 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 19:01:11.721482 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.721460 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hlmwm"] Jul 15 19:01:11.721577 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.721542 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 19:01:11.721762 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.721745 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jvmhf\"" Jul 15 19:01:11.722296 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.722275 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 19:01:11.732608 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.732589 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 19:01:11.785901 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.785813 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.785901 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.785878 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.785907 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzpc8\" (UniqueName: \"kubernetes.io/projected/ec80b08a-052a-4f8c-9536-fcb0121936cb-kube-api-access-tzpc8\") pod \"volume-data-source-validator-78495597f6-tgvwf\" (UID: \"ec80b08a-052a-4f8c-9536-fcb0121936cb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.785968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf3ec3be-080b-4691-838d-2a72b6e7192e-trusted-ca\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.785983 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.785993 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg4p4\" (UniqueName: \"kubernetes.io/projected/cf3ec3be-080b-4691-838d-2a72b6e7192e-kube-api-access-qg4p4\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.786050 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls podName:613706c9-971d-4afb-bec1-093b211a3b51 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:12.286035506 +0000 UTC m=+41.753009800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wgxbr" (UID: "613706c9-971d-4afb-bec1-093b211a3b51") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/613706c9-971d-4afb-bec1-093b211a3b51-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786100 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf3ec3be-080b-4691-838d-2a72b6e7192e-config\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.786129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786124 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-default-certificate\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786150 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786176 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-stats-auth\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786203 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8228z\" (UniqueName: \"kubernetes.io/projected/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-kube-api-access-8228z\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786249 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwts7\" (UniqueName: \"kubernetes.io/projected/613706c9-971d-4afb-bec1-093b211a3b51-kube-api-access-rwts7\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786293 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf3ec3be-080b-4691-838d-2a72b6e7192e-serving-cert\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786325 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd9bm\" (UniqueName: \"kubernetes.io/projected/b656aa6d-2d76-41d4-a398-cc3bfc725624-kube-api-access-gd9bm\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.786539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786351 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.786954 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.786933 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/613706c9-971d-4afb-bec1-093b211a3b51-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.819695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.819665 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwts7\" (UniqueName: \"kubernetes.io/projected/613706c9-971d-4afb-bec1-093b211a3b51-kube-api-access-rwts7\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:11.886820 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.886788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf3ec3be-080b-4691-838d-2a72b6e7192e-config\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.886820 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.886825 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-default-certificate\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.887047 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.886856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:11.887047 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.886884 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-stats-auth\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.887047 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.886907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8228z\" (UniqueName: \"kubernetes.io/projected/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-kube-api-access-8228z\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:11.887047 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.887007 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:01:11.887260 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.887079 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls podName:6b8b1082-bbcf-43e2-9af8-3f9575c46c48 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:12.387057777 +0000 UTC m=+41.854032082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls") pod "cluster-samples-operator-65db858894-dv4wf" (UID: "6b8b1082-bbcf-43e2-9af8-3f9575c46c48") : secret "samples-operator-tls" not found Jul 15 19:01:11.887337 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887282 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3467ed9c-093c-4b71-b519-3dffa22383eb-serving-cert\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.887337 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887321 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3467ed9c-093c-4b71-b519-3dffa22383eb-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.887431 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887367 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf3ec3be-080b-4691-838d-2a72b6e7192e-serving-cert\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.887431 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887398 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3467ed9c-093c-4b71-b519-3dffa22383eb-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.887587 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887439 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6nps\" (UniqueName: \"kubernetes.io/projected/3467ed9c-093c-4b71-b519-3dffa22383eb-kube-api-access-x6nps\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.887587 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887479 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd9bm\" (UniqueName: \"kubernetes.io/projected/b656aa6d-2d76-41d4-a398-cc3bfc725624-kube-api-access-gd9bm\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.887587 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887512 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.887733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887590 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3467ed9c-093c-4b71-b519-3dffa22383eb-snapshots\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.887733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887590 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf3ec3be-080b-4691-838d-2a72b6e7192e-config\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.887733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887630 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.887733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpc8\" (UniqueName: \"kubernetes.io/projected/ec80b08a-052a-4f8c-9536-fcb0121936cb-kube-api-access-tzpc8\") pod \"volume-data-source-validator-78495597f6-tgvwf\" (UID: \"ec80b08a-052a-4f8c-9536-fcb0121936cb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" Jul 15 19:01:11.887733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887721 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3467ed9c-093c-4b71-b519-3dffa22383eb-tmp\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.887964 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887752 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf3ec3be-080b-4691-838d-2a72b6e7192e-trusted-ca\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.887964 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.887779 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg4p4\" (UniqueName: \"kubernetes.io/projected/cf3ec3be-080b-4691-838d-2a72b6e7192e-kube-api-access-qg4p4\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.887964 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.887923 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:01:11.888099 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.887977 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:12.387959947 +0000 UTC m=+41.854934239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : secret "router-metrics-certs-default" not found Jul 15 19:01:11.888099 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:11.888061 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:12.388048735 +0000 UTC m=+41.855023046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : configmap references non-existent config key: service-ca.crt Jul 15 19:01:11.889144 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.889121 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf3ec3be-080b-4691-838d-2a72b6e7192e-trusted-ca\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.889708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.889662 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-default-certificate\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.889816 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.889713 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-stats-auth\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.889903 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.889883 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf3ec3be-080b-4691-838d-2a72b6e7192e-serving-cert\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.899299 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.899274 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8228z\" (UniqueName: \"kubernetes.io/projected/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-kube-api-access-8228z\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:11.899704 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.899673 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzpc8\" (UniqueName: \"kubernetes.io/projected/ec80b08a-052a-4f8c-9536-fcb0121936cb-kube-api-access-tzpc8\") pod \"volume-data-source-validator-78495597f6-tgvwf\" (UID: \"ec80b08a-052a-4f8c-9536-fcb0121936cb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" Jul 15 19:01:11.900343 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.900320 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg4p4\" (UniqueName: \"kubernetes.io/projected/cf3ec3be-080b-4691-838d-2a72b6e7192e-kube-api-access-qg4p4\") pod \"console-operator-7866497494-2kz84\" (UID: \"cf3ec3be-080b-4691-838d-2a72b6e7192e\") " pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.900539 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.900519 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd9bm\" (UniqueName: \"kubernetes.io/projected/b656aa6d-2d76-41d4-a398-cc3bfc725624-kube-api-access-gd9bm\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:11.931062 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.931033 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" Jul 15 19:01:11.964317 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.964293 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:11.989062 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989035 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3467ed9c-093c-4b71-b519-3dffa22383eb-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.989144 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6nps\" (UniqueName: \"kubernetes.io/projected/3467ed9c-093c-4b71-b519-3dffa22383eb-kube-api-access-x6nps\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.989302 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3467ed9c-093c-4b71-b519-3dffa22383eb-snapshots\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.989398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989383 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3467ed9c-093c-4b71-b519-3dffa22383eb-tmp\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.989482 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989455 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3467ed9c-093c-4b71-b519-3dffa22383eb-serving-cert\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.989586 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3467ed9c-093c-4b71-b519-3dffa22383eb-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.989734 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989678 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3467ed9c-093c-4b71-b519-3dffa22383eb-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.990010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989965 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3467ed9c-093c-4b71-b519-3dffa22383eb-tmp\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.990086 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.989979 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3467ed9c-093c-4b71-b519-3dffa22383eb-snapshots\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.990785 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.990764 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3467ed9c-093c-4b71-b519-3dffa22383eb-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.992109 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.992087 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3467ed9c-093c-4b71-b519-3dffa22383eb-serving-cert\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:11.998328 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:11.998306 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6nps\" (UniqueName: \"kubernetes.io/projected/3467ed9c-093c-4b71-b519-3dffa22383eb-kube-api-access-x6nps\") pod \"insights-operator-68d9dd94cf-hlmwm\" (UID: \"3467ed9c-093c-4b71-b519-3dffa22383eb\") " pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:12.027195 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.027163 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" Jul 15 19:01:12.292566 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.292522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:12.292765 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.292664 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:12.292765 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.292752 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls podName:613706c9-971d-4afb-bec1-093b211a3b51 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:13.292731748 +0000 UTC m=+42.759706040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wgxbr" (UID: "613706c9-971d-4afb-bec1-093b211a3b51") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:12.393129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.393095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:12.393299 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.393170 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:12.393299 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.393227 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:12.393432 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.393345 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:13.393319895 +0000 UTC m=+42.860294199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : configmap references non-existent config key: service-ca.crt Jul 15 19:01:12.393432 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.393372 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:01:12.393432 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.393428 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:13.39341045 +0000 UTC m=+42.860384754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : secret "router-metrics-certs-default" not found Jul 15 19:01:12.393590 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.393374 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:01:12.393590 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:12.393474 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls podName:6b8b1082-bbcf-43e2-9af8-3f9575c46c48 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:13.393465031 +0000 UTC m=+42.860439333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls") pod "cluster-samples-operator-65db858894-dv4wf" (UID: "6b8b1082-bbcf-43e2-9af8-3f9575c46c48") : secret "samples-operator-tls" not found Jul 15 19:01:12.528175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.528105 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf"] Jul 15 19:01:12.531756 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:12.531721 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec80b08a_052a_4f8c_9536_fcb0121936cb.slice/crio-b22c2983f88beb35d1731ee214e8378a56086b60f18a2dcc5cd6b65d5024fad1 WatchSource:0}: Error finding container b22c2983f88beb35d1731ee214e8378a56086b60f18a2dcc5cd6b65d5024fad1: Status 404 returned error can't find the container with id b22c2983f88beb35d1731ee214e8378a56086b60f18a2dcc5cd6b65d5024fad1 Jul 15 19:01:12.545772 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.545745 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-2kz84"] Jul 15 19:01:12.560885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:12.560864 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hlmwm"] Jul 15 19:01:12.719111 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:12.719042 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3ec3be_080b_4691_838d_2a72b6e7192e.slice/crio-4d16752f7fc3671e51cb7216d1555611b06f3946372547ce99a2cce59ee8ff3f WatchSource:0}: Error finding container 4d16752f7fc3671e51cb7216d1555611b06f3946372547ce99a2cce59ee8ff3f: Status 404 returned error can't find the container with id 4d16752f7fc3671e51cb7216d1555611b06f3946372547ce99a2cce59ee8ff3f Jul 15 19:01:12.719752 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:12.719725 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3467ed9c_093c_4b71_b519_3dffa22383eb.slice/crio-548c1485da467f47ea9cca4b5c1c222a284a367a045a5888fd8ebae030177d33 WatchSource:0}: Error finding container 548c1485da467f47ea9cca4b5c1c222a284a367a045a5888fd8ebae030177d33: Status 404 returned error can't find the container with id 548c1485da467f47ea9cca4b5c1c222a284a367a045a5888fd8ebae030177d33 Jul 15 19:01:13.247411 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.247315 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-2kz84" event={"ID":"cf3ec3be-080b-4691-838d-2a72b6e7192e","Type":"ContainerStarted","Data":"4d16752f7fc3671e51cb7216d1555611b06f3946372547ce99a2cce59ee8ff3f"} Jul 15 19:01:13.248806 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.248776 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" event={"ID":"3467ed9c-093c-4b71-b519-3dffa22383eb","Type":"ContainerStarted","Data":"548c1485da467f47ea9cca4b5c1c222a284a367a045a5888fd8ebae030177d33"} Jul 15 19:01:13.250123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.250094 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" event={"ID":"ec80b08a-052a-4f8c-9536-fcb0121936cb","Type":"ContainerStarted","Data":"b22c2983f88beb35d1731ee214e8378a56086b60f18a2dcc5cd6b65d5024fad1"} Jul 15 19:01:13.253423 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.253399 2572 generic.go:356] "Generic (PLEG): container finished" podID="94e5f207-84e3-4d24-8b1d-82bfd25e0c25" containerID="d40cb580aa40c4aeee0bb37c6a2d30250b9ea97b463f4ef5c05026c30e1d5f01" exitCode=0 Jul 15 19:01:13.253527 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.253446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerDied","Data":"d40cb580aa40c4aeee0bb37c6a2d30250b9ea97b463f4ef5c05026c30e1d5f01"} Jul 15 19:01:13.300604 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.300571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:13.300729 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.300711 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:13.300789 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.300778 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls podName:613706c9-971d-4afb-bec1-093b211a3b51 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:15.300762489 +0000 UTC m=+44.767736802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wgxbr" (UID: "613706c9-971d-4afb-bec1-093b211a3b51") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:13.401588 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.401555 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:13.401691 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.401651 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:13.401745 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.401701 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:01:13.401792 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.401748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:13.401792 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.401766 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls podName:6b8b1082-bbcf-43e2-9af8-3f9575c46c48 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:15.401746415 +0000 UTC m=+44.868720722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls") pod "cluster-samples-operator-65db858894-dv4wf" (UID: "6b8b1082-bbcf-43e2-9af8-3f9575c46c48") : secret "samples-operator-tls" not found Jul 15 19:01:13.401950 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.401832 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:01:13.401950 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.401871 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:15.40184925 +0000 UTC m=+44.868823540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : configmap references non-existent config key: service-ca.crt Jul 15 19:01:13.401950 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:13.401902 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:15.401890036 +0000 UTC m=+44.868864326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : secret "router-metrics-certs-default" not found Jul 15 19:01:13.478192 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.478162 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl"] Jul 15 19:01:13.496883 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.496835 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl"] Jul 15 19:01:13.497028 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.496984 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.500014 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.499950 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:01:13.500120 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.500030 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 19:01:13.500120 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.500041 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-24b4q\"" Jul 15 19:01:13.500207 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.500128 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 19:01:13.500381 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.500364 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 19:01:13.602982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.602943 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/176a3fce-370b-48ee-b6b2-71302f9d3731-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.603386 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.603008 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfrp4\" (UniqueName: \"kubernetes.io/projected/176a3fce-370b-48ee-b6b2-71302f9d3731-kube-api-access-tfrp4\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.603386 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.603069 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/176a3fce-370b-48ee-b6b2-71302f9d3731-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.704372 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.704336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/176a3fce-370b-48ee-b6b2-71302f9d3731-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.704534 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.704395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfrp4\" (UniqueName: \"kubernetes.io/projected/176a3fce-370b-48ee-b6b2-71302f9d3731-kube-api-access-tfrp4\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.704594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.704542 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/176a3fce-370b-48ee-b6b2-71302f9d3731-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.705069 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.705052 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/176a3fce-370b-48ee-b6b2-71302f9d3731-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.706486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.706472 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/176a3fce-370b-48ee-b6b2-71302f9d3731-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.713004 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.712972 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfrp4\" (UniqueName: \"kubernetes.io/projected/176a3fce-370b-48ee-b6b2-71302f9d3731-kube-api-access-tfrp4\") pod \"kube-storage-version-migrator-operator-799cb7f958-fjxtl\" (UID: \"176a3fce-370b-48ee-b6b2-71302f9d3731\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.808532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.808454 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" Jul 15 19:01:13.930190 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:13.930151 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl"] Jul 15 19:01:14.028123 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:14.028082 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176a3fce_370b_48ee_b6b2_71302f9d3731.slice/crio-bc0c75097931701da1a6edafb56cbee3774236f1dafb606e5fdb50d16168ad05 WatchSource:0}: Error finding container bc0c75097931701da1a6edafb56cbee3774236f1dafb606e5fdb50d16168ad05: Status 404 returned error can't find the container with id bc0c75097931701da1a6edafb56cbee3774236f1dafb606e5fdb50d16168ad05 Jul 15 19:01:14.208896 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.208867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:14.209092 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:14.209014 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:01:14.209092 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.209073 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:14.209092 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:14.209084 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls podName:9de406e7-cfc2-4a52-b55b-c7819d750e2a nodeName:}" failed. No retries permitted until 2026-07-15 19:01:22.209066009 +0000 UTC m=+51.676040302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls") pod "dns-default-z65bn" (UID: "9de406e7-cfc2-4a52-b55b-c7819d750e2a") : secret "dns-default-metrics-tls" not found Jul 15 19:01:14.209270 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:14.209227 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:01:14.209270 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:14.209262 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5d67c5b9-sjnnt: secret "image-registry-tls" not found Jul 15 19:01:14.209368 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:14.209318 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls podName:361939d2-4fef-45ba-9488-1b42cc04c7a9 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:22.209300864 +0000 UTC m=+51.676275170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls") pod "image-registry-5b5d67c5b9-sjnnt" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9") : secret "image-registry-tls" not found Jul 15 19:01:14.258546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.258515 2572 generic.go:356] "Generic (PLEG): container finished" podID="94e5f207-84e3-4d24-8b1d-82bfd25e0c25" containerID="a517718f639b52e6c415c3df4d46a76e19fcd695a0724352244bed1e6b9f1f8f" exitCode=0 Jul 15 19:01:14.258708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.258594 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerDied","Data":"a517718f639b52e6c415c3df4d46a76e19fcd695a0724352244bed1e6b9f1f8f"} Jul 15 19:01:14.259723 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.259696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" event={"ID":"176a3fce-370b-48ee-b6b2-71302f9d3731","Type":"ContainerStarted","Data":"bc0c75097931701da1a6edafb56cbee3774236f1dafb606e5fdb50d16168ad05"} Jul 15 19:01:14.261008 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.260988 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" event={"ID":"ec80b08a-052a-4f8c-9536-fcb0121936cb","Type":"ContainerStarted","Data":"4fdea689ce6388b5dd585098d2aa3c1099fdbe95fad01936f63c0cea0ae8d3a0"} Jul 15 19:01:14.298558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:14.298501 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-tgvwf" podStartSLOduration=1.763121007 podStartE2EDuration="3.29848166s" totalImagesPullingTime="1.535360653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:14.297186888 +0000 UTC m=+43.764161201" watchObservedRunningTime="2026-07-15 19:01:14.29848166 +0000 UTC m=+43.765455952" Jul 15 19:01:15.268909 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.268869 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" event={"ID":"94e5f207-84e3-4d24-8b1d-82bfd25e0c25","Type":"ContainerStarted","Data":"82b89eb602e3bf069c118ee99cd2671919f82c53d34182ba71cd66e162c3457b"} Jul 15 19:01:15.294633 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.294569 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vb4xn" podStartSLOduration=7.684700483 podStartE2EDuration="43.294499947s" totalImagesPullingTime="35.609799464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:15.292609451 +0000 UTC m=+44.759583793" watchObservedRunningTime="2026-07-15 19:01:15.294499947 +0000 UTC m=+44.761474259" Jul 15 19:01:15.320221 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.320117 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:15.320397 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.320285 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:15.320397 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.320366 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls podName:613706c9-971d-4afb-bec1-093b211a3b51 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:19.320344305 +0000 UTC m=+48.787318609 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wgxbr" (UID: "613706c9-971d-4afb-bec1-093b211a3b51") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:15.421373 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.421340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:15.421556 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.421427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:15.421556 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.421472 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:01:15.421556 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.421490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:15.421556 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.421548 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls podName:6b8b1082-bbcf-43e2-9af8-3f9575c46c48 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:19.421527559 +0000 UTC m=+48.888501864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls") pod "cluster-samples-operator-65db858894-dv4wf" (UID: "6b8b1082-bbcf-43e2-9af8-3f9575c46c48") : secret "samples-operator-tls" not found Jul 15 19:01:15.422100 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.422078 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:01:15.422220 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.422134 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:19.422118839 +0000 UTC m=+48.889093133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : secret "router-metrics-certs-default" not found Jul 15 19:01:15.422379 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:15.422356 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:19.422343556 +0000 UTC m=+48.889317845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : configmap references non-existent config key: service-ca.crt Jul 15 19:01:15.530475 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.530389 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh"] Jul 15 19:01:15.547692 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.547658 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh"] Jul 15 19:01:15.547854 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.547787 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.550581 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.550550 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 19:01:15.550733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.550602 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:01:15.550733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.550687 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4hpb6\"" Jul 15 19:01:15.550864 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.550755 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 19:01:15.551002 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.550986 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 19:01:15.723687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.723654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4731e8b5-29d8-461e-8918-456eac75f5a8-config\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.723864 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.723702 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4731e8b5-29d8-461e-8918-456eac75f5a8-serving-cert\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.723864 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.723746 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whb75\" (UniqueName: \"kubernetes.io/projected/4731e8b5-29d8-461e-8918-456eac75f5a8-kube-api-access-whb75\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.815803 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.815723 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl"] Jul 15 19:01:15.824734 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.824704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whb75\" (UniqueName: \"kubernetes.io/projected/4731e8b5-29d8-461e-8918-456eac75f5a8-kube-api-access-whb75\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.824879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.824842 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4731e8b5-29d8-461e-8918-456eac75f5a8-config\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.824938 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.824891 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4731e8b5-29d8-461e-8918-456eac75f5a8-serving-cert\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.825587 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.825559 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4731e8b5-29d8-461e-8918-456eac75f5a8-config\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.828074 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.828054 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4731e8b5-29d8-461e-8918-456eac75f5a8-serving-cert\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.832976 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.832954 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl"] Jul 15 19:01:15.833106 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.833089 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" Jul 15 19:01:15.835689 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.835668 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fnn7l\"" Jul 15 19:01:15.836853 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.836828 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whb75\" (UniqueName: \"kubernetes.io/projected/4731e8b5-29d8-461e-8918-456eac75f5a8-kube-api-access-whb75\") pod \"service-ca-operator-7695fc6c8c-bfxfh\" (UID: \"4731e8b5-29d8-461e-8918-456eac75f5a8\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.858979 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.858953 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" Jul 15 19:01:15.926300 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:15.926267 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r59pm\" (UniqueName: \"kubernetes.io/projected/c2c5d95c-2b95-41a1-ba2a-a57423f3e51b-kube-api-access-r59pm\") pod \"network-check-source-869dcfcdcd-9q6bl\" (UID: \"c2c5d95c-2b95-41a1-ba2a-a57423f3e51b\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" Jul 15 19:01:16.027496 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:16.027460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r59pm\" (UniqueName: \"kubernetes.io/projected/c2c5d95c-2b95-41a1-ba2a-a57423f3e51b-kube-api-access-r59pm\") pod \"network-check-source-869dcfcdcd-9q6bl\" (UID: \"c2c5d95c-2b95-41a1-ba2a-a57423f3e51b\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" Jul 15 19:01:16.036411 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:16.036376 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r59pm\" (UniqueName: \"kubernetes.io/projected/c2c5d95c-2b95-41a1-ba2a-a57423f3e51b-kube-api-access-r59pm\") pod \"network-check-source-869dcfcdcd-9q6bl\" (UID: \"c2c5d95c-2b95-41a1-ba2a-a57423f3e51b\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" Jul 15 19:01:16.153009 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:16.152971 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" Jul 15 19:01:17.631887 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:17.631857 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh"] Jul 15 19:01:17.634767 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:17.634736 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4731e8b5_29d8_461e_8918_456eac75f5a8.slice/crio-b4c52229dbabeb4673fa6bc16af7bbd7d0c20d82e4c719b8ef2ffa25baf69f81 WatchSource:0}: Error finding container b4c52229dbabeb4673fa6bc16af7bbd7d0c20d82e4c719b8ef2ffa25baf69f81: Status 404 returned error can't find the container with id b4c52229dbabeb4673fa6bc16af7bbd7d0c20d82e4c719b8ef2ffa25baf69f81 Jul 15 19:01:17.645543 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:17.645518 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl"] Jul 15 19:01:17.649420 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:17.649392 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c5d95c_2b95_41a1_ba2a_a57423f3e51b.slice/crio-15541593dcbf9a9843f5aa73c634142d28ad4022759a7ede952ae7d71508323c WatchSource:0}: Error finding container 15541593dcbf9a9843f5aa73c634142d28ad4022759a7ede952ae7d71508323c: Status 404 returned error can't find the container with id 15541593dcbf9a9843f5aa73c634142d28ad4022759a7ede952ae7d71508323c Jul 15 19:01:18.279474 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.279393 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/0.log" Jul 15 19:01:18.279474 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.279432 2572 generic.go:356] "Generic (PLEG): container finished" podID="cf3ec3be-080b-4691-838d-2a72b6e7192e" containerID="64b8b4d904bfe0fbfcfa68b7cb38b6a2fdff4a3b89a981469409732af859bc65" exitCode=255 Jul 15 19:01:18.279675 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.279506 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-2kz84" event={"ID":"cf3ec3be-080b-4691-838d-2a72b6e7192e","Type":"ContainerDied","Data":"64b8b4d904bfe0fbfcfa68b7cb38b6a2fdff4a3b89a981469409732af859bc65"} Jul 15 19:01:18.279763 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.279745 2572 scope.go:117] "RemoveContainer" containerID="64b8b4d904bfe0fbfcfa68b7cb38b6a2fdff4a3b89a981469409732af859bc65" Jul 15 19:01:18.281267 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.281224 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" event={"ID":"176a3fce-370b-48ee-b6b2-71302f9d3731","Type":"ContainerStarted","Data":"64d594819bb599ba7cff789d45b611adb8f75155e39ed1af6307746989eeaba3"} Jul 15 19:01:18.282602 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.282569 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" event={"ID":"4731e8b5-29d8-461e-8918-456eac75f5a8","Type":"ContainerStarted","Data":"b4c52229dbabeb4673fa6bc16af7bbd7d0c20d82e4c719b8ef2ffa25baf69f81"} Jul 15 19:01:18.284127 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.284104 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" event={"ID":"3467ed9c-093c-4b71-b519-3dffa22383eb","Type":"ContainerStarted","Data":"ba904b67b79c3910163c1cbe4ff2a9a28defee8def70e46fe94bba4f4f02f875"} Jul 15 19:01:18.285261 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.285221 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" event={"ID":"c2c5d95c-2b95-41a1-ba2a-a57423f3e51b","Type":"ContainerStarted","Data":"15541593dcbf9a9843f5aa73c634142d28ad4022759a7ede952ae7d71508323c"} Jul 15 19:01:18.314611 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.314568 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" podStartSLOduration=2.56545274 podStartE2EDuration="7.314552624s" totalImagesPullingTime="4.749099884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:18.31432968 +0000 UTC m=+47.781303990" watchObservedRunningTime="2026-07-15 19:01:18.314552624 +0000 UTC m=+47.781526935" Jul 15 19:01:18.330380 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.330330 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" podStartSLOduration=1.477049134 podStartE2EDuration="5.330311511s" totalImagesPullingTime="3.853262377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:18.329064631 +0000 UTC m=+47.796038943" watchObservedRunningTime="2026-07-15 19:01:18.330311511 +0000 UTC m=+47.797285872" Jul 15 19:01:18.905169 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.905055 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xlj2j"] Jul 15 19:01:18.925693 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.925665 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xlj2j"] Jul 15 19:01:18.925861 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.925798 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:18.928559 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.928536 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tn2w9\"" Jul 15 19:01:18.928687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.928535 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 19:01:18.928687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.928535 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 19:01:18.957837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.957801 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:18.957988 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.957931 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zwx6\" (UniqueName: \"kubernetes.io/projected/183b0d00-adc2-4d7a-806b-916a026874f1-kube-api-access-4zwx6\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:18.958058 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.958038 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/183b0d00-adc2-4d7a-806b-916a026874f1-crio-socket\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:18.958107 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.958070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:18.958214 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:18.958188 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/183b0d00-adc2-4d7a-806b-916a026874f1-data-volume\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.058645 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.058609 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/183b0d00-adc2-4d7a-806b-916a026874f1-crio-socket\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.058828 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.058648 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.058828 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.058756 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/183b0d00-adc2-4d7a-806b-916a026874f1-data-volume\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.058828 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.058757 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:01:19.058828 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.058807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.059043 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.058848 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:19.558826199 +0000 UTC m=+49.025800489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : configmap "kube-rbac-proxy" not found Jul 15 19:01:19.059043 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.058903 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:01:19.059043 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.058917 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zwx6\" (UniqueName: \"kubernetes.io/projected/183b0d00-adc2-4d7a-806b-916a026874f1-kube-api-access-4zwx6\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.059043 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.058955 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:19.558939177 +0000 UTC m=+49.025913467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : secret "insights-runtime-extractor-tls" not found Jul 15 19:01:19.059272 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.059168 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/183b0d00-adc2-4d7a-806b-916a026874f1-data-volume\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.059403 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.059373 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/183b0d00-adc2-4d7a-806b-916a026874f1-crio-socket\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.068867 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.068831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zwx6\" (UniqueName: \"kubernetes.io/projected/183b0d00-adc2-4d7a-806b-916a026874f1-kube-api-access-4zwx6\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.291637 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.291549 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/1.log" Jul 15 19:01:19.292615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.292591 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/0.log" Jul 15 19:01:19.292743 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.292633 2572 generic.go:356] "Generic (PLEG): container finished" podID="cf3ec3be-080b-4691-838d-2a72b6e7192e" containerID="f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe" exitCode=255 Jul 15 19:01:19.293040 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.292876 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-2kz84" event={"ID":"cf3ec3be-080b-4691-838d-2a72b6e7192e","Type":"ContainerDied","Data":"f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe"} Jul 15 19:01:19.293040 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.292984 2572 scope.go:117] "RemoveContainer" containerID="64b8b4d904bfe0fbfcfa68b7cb38b6a2fdff4a3b89a981469409732af859bc65" Jul 15 19:01:19.293407 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.293389 2572 scope.go:117] "RemoveContainer" containerID="f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe" Jul 15 19:01:19.293628 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.293602 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-2kz84_openshift-console-operator(cf3ec3be-080b-4691-838d-2a72b6e7192e)\"" pod="openshift-console-operator/console-operator-7866497494-2kz84" podUID="cf3ec3be-080b-4691-838d-2a72b6e7192e" Jul 15 19:01:19.362203 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.362149 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:19.362382 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.362348 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:19.362441 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.362409 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls podName:613706c9-971d-4afb-bec1-093b211a3b51 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:27.362390298 +0000 UTC m=+56.829364590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wgxbr" (UID: "613706c9-971d-4afb-bec1-093b211a3b51") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:19.463566 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.463513 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:19.464197 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.463845 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:27.463822854 +0000 UTC m=+56.930797166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : configmap references non-existent config key: service-ca.crt Jul 15 19:01:19.464197 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.464141 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh"] Jul 15 19:01:19.464529 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.464503 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:19.464649 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.464623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:19.464898 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.464792 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:01:19.464898 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.464852 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls podName:6b8b1082-bbcf-43e2-9af8-3f9575c46c48 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:27.464839073 +0000 UTC m=+56.931813365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls") pod "cluster-samples-operator-65db858894-dv4wf" (UID: "6b8b1082-bbcf-43e2-9af8-3f9575c46c48") : secret "samples-operator-tls" not found Jul 15 19:01:19.465060 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.464910 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:01:19.465060 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.464942 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:27.464931772 +0000 UTC m=+56.931906077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : secret "router-metrics-certs-default" not found Jul 15 19:01:19.492375 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.492311 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh"] Jul 15 19:01:19.492535 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.492449 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" Jul 15 19:01:19.495757 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.495527 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 19:01:19.495757 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.495555 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 19:01:19.495757 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.495559 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b8j8s\"" Jul 15 19:01:19.566025 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.565943 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.566182 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.566070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7fb\" (UniqueName: \"kubernetes.io/projected/1a17900a-1ce3-4955-8601-9cf3efe6b24d-kube-api-access-zq7fb\") pod \"migrator-7f59b54f6c-hh4jh\" (UID: \"1a17900a-1ce3-4955-8601-9cf3efe6b24d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" Jul 15 19:01:19.566182 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.566101 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:19.566182 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.566129 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:01:19.566182 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.566172 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:01:19.566413 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.566206 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:20.566184017 +0000 UTC m=+50.033158328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : secret "insights-runtime-extractor-tls" not found Jul 15 19:01:19.566413 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:19.566278 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:20.566256915 +0000 UTC m=+50.033231207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : configmap "kube-rbac-proxy" not found Jul 15 19:01:19.667508 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.667453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7fb\" (UniqueName: \"kubernetes.io/projected/1a17900a-1ce3-4955-8601-9cf3efe6b24d-kube-api-access-zq7fb\") pod \"migrator-7f59b54f6c-hh4jh\" (UID: \"1a17900a-1ce3-4955-8601-9cf3efe6b24d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" Jul 15 19:01:19.677591 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.677555 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7fb\" (UniqueName: \"kubernetes.io/projected/1a17900a-1ce3-4955-8601-9cf3efe6b24d-kube-api-access-zq7fb\") pod \"migrator-7f59b54f6c-hh4jh\" (UID: \"1a17900a-1ce3-4955-8601-9cf3efe6b24d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" Jul 15 19:01:19.804620 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.804580 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" Jul 15 19:01:19.907151 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.907118 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2npnt_6196293c-156c-4f46-9f14-f228ec4b2faa/dns-node-resolver/0.log" Jul 15 19:01:19.974184 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:19.974075 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh"] Jul 15 19:01:19.978437 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:19.978410 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a17900a_1ce3_4955_8601_9cf3efe6b24d.slice/crio-b8937c35f89876111d4f2b2a14deb78cbd2b589549ebc19c0f42785e0b6a0d2f WatchSource:0}: Error finding container b8937c35f89876111d4f2b2a14deb78cbd2b589549ebc19c0f42785e0b6a0d2f: Status 404 returned error can't find the container with id b8937c35f89876111d4f2b2a14deb78cbd2b589549ebc19c0f42785e0b6a0d2f Jul 15 19:01:20.299090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:20.299057 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/1.log" Jul 15 19:01:20.299553 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:20.299524 2572 scope.go:117] "RemoveContainer" containerID="f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe" Jul 15 19:01:20.299753 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:20.299727 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-2kz84_openshift-console-operator(cf3ec3be-080b-4691-838d-2a72b6e7192e)\"" pod="openshift-console-operator/console-operator-7866497494-2kz84" podUID="cf3ec3be-080b-4691-838d-2a72b6e7192e" Jul 15 19:01:20.301054 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:20.301010 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" event={"ID":"1a17900a-1ce3-4955-8601-9cf3efe6b24d","Type":"ContainerStarted","Data":"b8937c35f89876111d4f2b2a14deb78cbd2b589549ebc19c0f42785e0b6a0d2f"} Jul 15 19:01:20.577098 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:20.577023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:20.577256 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:20.577144 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:20.577256 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:20.577215 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:01:20.577373 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:20.577278 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:01:20.577373 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:20.577315 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:22.577294668 +0000 UTC m=+52.044268962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : secret "insights-runtime-extractor-tls" not found Jul 15 19:01:20.577373 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:20.577340 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:22.577328519 +0000 UTC m=+52.044302811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : configmap "kube-rbac-proxy" not found Jul 15 19:01:20.706656 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:20.706625 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dz2bk_c6d5c01e-3cbb-476f-b39c-9960a8e4beb7/node-ca/0.log" Jul 15 19:01:21.306765 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:21.306711 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c79mn_dba7f537-3e9b-41bc-ba46-cf83477ec8a4/serve-healthcheck-canary/0.log" Jul 15 19:01:21.965271 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:21.965224 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:21.965435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:21.965316 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:21.965677 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:21.965665 2572 scope.go:117] "RemoveContainer" containerID="f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe" Jul 15 19:01:21.965863 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:21.965838 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-2kz84_openshift-console-operator(cf3ec3be-080b-4691-838d-2a72b6e7192e)\"" pod="openshift-console-operator/console-operator-7866497494-2kz84" podUID="cf3ec3be-080b-4691-838d-2a72b6e7192e" Jul 15 19:01:22.293089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:22.293007 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:22.293210 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:22.293093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:22.293210 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.293159 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:01:22.293210 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.293183 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:01:22.293210 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.293192 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5d67c5b9-sjnnt: secret "image-registry-tls" not found Jul 15 19:01:22.293456 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.293224 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls podName:9de406e7-cfc2-4a52-b55b-c7819d750e2a nodeName:}" failed. No retries permitted until 2026-07-15 19:01:38.293207393 +0000 UTC m=+67.760181685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls") pod "dns-default-z65bn" (UID: "9de406e7-cfc2-4a52-b55b-c7819d750e2a") : secret "dns-default-metrics-tls" not found Jul 15 19:01:22.293456 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.293258 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls podName:361939d2-4fef-45ba-9488-1b42cc04c7a9 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:38.293250333 +0000 UTC m=+67.760224622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls") pod "image-registry-5b5d67c5b9-sjnnt" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9") : secret "image-registry-tls" not found Jul 15 19:01:22.595605 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:22.595516 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:22.596005 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.595706 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:01:22.596005 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:22.595722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:22.596005 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.595785 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:26.595764422 +0000 UTC m=+56.062738731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : secret "insights-runtime-extractor-tls" not found Jul 15 19:01:22.596005 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.595816 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:01:22.596005 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:22.595872 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:26.595857788 +0000 UTC m=+56.062832094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : configmap "kube-rbac-proxy" not found Jul 15 19:01:23.313396 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:23.313363 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" event={"ID":"4731e8b5-29d8-461e-8918-456eac75f5a8","Type":"ContainerStarted","Data":"864fe569c4da3e402b32f17e236c080f8909f2e1d8123f369951af104b6dd171"} Jul 15 19:01:23.315058 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:23.315018 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" event={"ID":"1a17900a-1ce3-4955-8601-9cf3efe6b24d","Type":"ContainerStarted","Data":"91070e052fee418407b3e50baf00a5b9b976c96de197a1db038da75397fab724"} Jul 15 19:01:23.316659 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:23.316581 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" event={"ID":"c2c5d95c-2b95-41a1-ba2a-a57423f3e51b","Type":"ContainerStarted","Data":"d4c2380e02a3744879c8035517d21e3aa3b4f9335a9fcce5563d6f05e934159f"} Jul 15 19:01:23.329834 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:23.329790 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" podStartSLOduration=3.14835523 podStartE2EDuration="8.329775578s" totalImagesPullingTime="5.181420348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:23.328789722 +0000 UTC m=+52.795764035" watchObservedRunningTime="2026-07-15 19:01:23.329775578 +0000 UTC m=+52.796749890" Jul 15 19:01:23.347160 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:23.347087 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-9q6bl" podStartSLOduration=3.170394225 podStartE2EDuration="8.347066286s" totalImagesPullingTime="5.176672061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:23.346472338 +0000 UTC m=+52.813446646" watchObservedRunningTime="2026-07-15 19:01:23.347066286 +0000 UTC m=+52.814040598" Jul 15 19:01:24.322579 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:24.322537 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" event={"ID":"1a17900a-1ce3-4955-8601-9cf3efe6b24d","Type":"ContainerStarted","Data":"6fad6d1aa557359b6abae8d26fe8894ae15aa964be278fc4b94015f5470cf78b"} Jul 15 19:01:24.342267 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:24.342201 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hh4jh" podStartSLOduration=2.1497948 podStartE2EDuration="5.342184809s" totalImagesPullingTime="3.192390009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:24.340121593 +0000 UTC m=+53.807095905" watchObservedRunningTime="2026-07-15 19:01:24.342184809 +0000 UTC m=+53.809159121" Jul 15 19:01:26.635086 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:26.635048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:26.635670 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:26.635146 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:26.635670 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:26.635221 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:01:26.635670 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:26.635325 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:34.635302923 +0000 UTC m=+64.102277226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : secret "insights-runtime-extractor-tls" not found Jul 15 19:01:26.635670 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:26.635646 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/183b0d00-adc2-4d7a-806b-916a026874f1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:27.442702 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:27.442661 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:27.442891 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.442801 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:27.442891 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.442863 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls podName:613706c9-971d-4afb-bec1-093b211a3b51 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:43.442847573 +0000 UTC m=+72.909821861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wgxbr" (UID: "613706c9-971d-4afb-bec1-093b211a3b51") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:01:27.543565 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:27.543525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:27.543769 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:27.543607 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:27.543769 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.543694 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:01:27.543769 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.543728 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:43.543706764 +0000 UTC m=+73.010681060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : configmap references non-existent config key: service-ca.crt Jul 15 19:01:27.543769 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.543758 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs podName:b656aa6d-2d76-41d4-a398-cc3bfc725624 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:43.543748568 +0000 UTC m=+73.010722862 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs") pod "router-default-6dffcc884d-qt2hp" (UID: "b656aa6d-2d76-41d4-a398-cc3bfc725624") : secret "router-metrics-certs-default" not found Jul 15 19:01:27.543977 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:27.543855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:27.544018 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.543991 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:01:27.544060 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:27.544050 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls podName:6b8b1082-bbcf-43e2-9af8-3f9575c46c48 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:43.544035353 +0000 UTC m=+73.011009645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls") pod "cluster-samples-operator-65db858894-dv4wf" (UID: "6b8b1082-bbcf-43e2-9af8-3f9575c46c48") : secret "samples-operator-tls" not found Jul 15 19:01:31.213484 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:31.213454 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fn4z6" Jul 15 19:01:33.028280 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.028252 2572 scope.go:117] "RemoveContainer" containerID="f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe" Jul 15 19:01:33.351384 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.351356 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:01:33.351765 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.351751 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/1.log" Jul 15 19:01:33.351817 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.351783 2572 generic.go:356] "Generic (PLEG): container finished" podID="cf3ec3be-080b-4691-838d-2a72b6e7192e" containerID="71b44f40ad1ed184c379ca0450ff1d166ec6a66126758d96fdd0a79ae23b1d57" exitCode=255 Jul 15 19:01:33.351874 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.351857 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-2kz84" event={"ID":"cf3ec3be-080b-4691-838d-2a72b6e7192e","Type":"ContainerDied","Data":"71b44f40ad1ed184c379ca0450ff1d166ec6a66126758d96fdd0a79ae23b1d57"} Jul 15 19:01:33.351926 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.351916 2572 scope.go:117] "RemoveContainer" containerID="f9456ebb9ae7a527d80f9b0e2fe83ecacf0e5ed8ab5349eb36b5558e92242efe" Jul 15 19:01:33.352228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:33.352195 2572 scope.go:117] "RemoveContainer" containerID="71b44f40ad1ed184c379ca0450ff1d166ec6a66126758d96fdd0a79ae23b1d57" Jul 15 19:01:33.352447 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:33.352423 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-2kz84_openshift-console-operator(cf3ec3be-080b-4691-838d-2a72b6e7192e)\"" pod="openshift-console-operator/console-operator-7866497494-2kz84" podUID="cf3ec3be-080b-4691-838d-2a72b6e7192e" Jul 15 19:01:34.356815 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:34.356788 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:01:34.703022 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:34.702935 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:34.703192 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:34.703107 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:01:34.703192 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:34.703184 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls podName:183b0d00-adc2-4d7a-806b-916a026874f1 nodeName:}" failed. No retries permitted until 2026-07-15 19:01:50.703163999 +0000 UTC m=+80.170138288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlj2j" (UID: "183b0d00-adc2-4d7a-806b-916a026874f1") : secret "insights-runtime-extractor-tls" not found Jul 15 19:01:35.813417 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:35.813376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:01:35.815975 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:35.815955 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:01:35.824293 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:35.824278 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 19:01:35.824375 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:35.824334 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs podName:8e8c81ac-0a8b-4100-8e3b-d0472538bcb5 nodeName:}" failed. No retries permitted until 2026-07-15 19:02:39.824318144 +0000 UTC m=+129.291292433 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs") pod "network-metrics-daemon-kt4sp" (UID: "8e8c81ac-0a8b-4100-8e3b-d0472538bcb5") : secret "metrics-daemon-secret" not found Jul 15 19:01:35.914642 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:35.914613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:35.917335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:35.917315 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:01:35.927733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:35.927706 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35b4663c-ea09-4be7-a31c-660e76c46799-original-pull-secret\") pod \"global-pull-secret-syncer-b4dg5\" (UID: \"35b4663c-ea09-4be7-a31c-660e76c46799\") " pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:36.015417 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.015380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:36.017912 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.017891 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh46v\" (UniqueName: \"kubernetes.io/projected/a1c74c7b-4982-4e82-a1a6-4f2819ac9f36-kube-api-access-rh46v\") pod \"network-check-target-2bmgw\" (UID: \"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36\") " pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:36.143361 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.143327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4dg5" Jul 15 19:01:36.152244 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.152208 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j7rjq\"" Jul 15 19:01:36.159785 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.159761 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:36.280030 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.279992 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4dg5"] Jul 15 19:01:36.283000 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:36.282965 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b4663c_ea09_4be7_a31c_660e76c46799.slice/crio-ca0b94bc1dca5c77f634af69bf29a72b3be4ca5715bf9c742a5b19c7618d12fd WatchSource:0}: Error finding container ca0b94bc1dca5c77f634af69bf29a72b3be4ca5715bf9c742a5b19c7618d12fd: Status 404 returned error can't find the container with id ca0b94bc1dca5c77f634af69bf29a72b3be4ca5715bf9c742a5b19c7618d12fd Jul 15 19:01:36.297163 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.297051 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bmgw"] Jul 15 19:01:36.299517 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:36.299495 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c74c7b_4982_4e82_a1a6_4f2819ac9f36.slice/crio-67519a5ea2a78480c6fc144873078a36399547061ec61dc6eaa0c00d673ffe77 WatchSource:0}: Error finding container 67519a5ea2a78480c6fc144873078a36399547061ec61dc6eaa0c00d673ffe77: Status 404 returned error can't find the container with id 67519a5ea2a78480c6fc144873078a36399547061ec61dc6eaa0c00d673ffe77 Jul 15 19:01:36.364277 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.364243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bmgw" event={"ID":"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36","Type":"ContainerStarted","Data":"1b9efc11457228ef11d18b8d7939ea21f6ecee8eaa7d6ffc8380bb290c1b31a4"} Jul 15 19:01:36.364415 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.364311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bmgw" event={"ID":"a1c74c7b-4982-4e82-a1a6-4f2819ac9f36","Type":"ContainerStarted","Data":"67519a5ea2a78480c6fc144873078a36399547061ec61dc6eaa0c00d673ffe77"} Jul 15 19:01:36.364415 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.364342 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:01:36.365314 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.365288 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b4dg5" event={"ID":"35b4663c-ea09-4be7-a31c-660e76c46799","Type":"ContainerStarted","Data":"ca0b94bc1dca5c77f634af69bf29a72b3be4ca5715bf9c742a5b19c7618d12fd"} Jul 15 19:01:36.382085 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:36.382044 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2bmgw" podStartSLOduration=64.382029778 podStartE2EDuration="1m4.382029778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:36.381370401 +0000 UTC m=+65.848344716" watchObservedRunningTime="2026-07-15 19:01:36.382029778 +0000 UTC m=+65.849004092" Jul 15 19:01:38.334458 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.334427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:38.334943 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.334489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:38.336753 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.336727 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de406e7-cfc2-4a52-b55b-c7819d750e2a-metrics-tls\") pod \"dns-default-z65bn\" (UID: \"9de406e7-cfc2-4a52-b55b-c7819d750e2a\") " pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:38.336885 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.336866 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"image-registry-5b5d67c5b9-sjnnt\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:38.497676 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.497650 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z9lvq\"" Jul 15 19:01:38.505968 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.505953 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:38.511336 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.511312 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dgrg9\"" Jul 15 19:01:38.519639 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.519620 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:38.640443 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.640412 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5d67c5b9-sjnnt"] Jul 15 19:01:38.642741 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:38.642707 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361939d2_4fef_45ba_9488_1b42cc04c7a9.slice/crio-0fd43a8973b43e6cb71f0d3f268df64508b2fda5f78d770bd06d8f7f0c61ce43 WatchSource:0}: Error finding container 0fd43a8973b43e6cb71f0d3f268df64508b2fda5f78d770bd06d8f7f0c61ce43: Status 404 returned error can't find the container with id 0fd43a8973b43e6cb71f0d3f268df64508b2fda5f78d770bd06d8f7f0c61ce43 Jul 15 19:01:38.653949 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:38.653926 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z65bn"] Jul 15 19:01:38.658188 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:38.658163 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de406e7_cfc2_4a52_b55b_c7819d750e2a.slice/crio-b0f16c3391419f1edb8a37f7ecc0299452bbbe114826d4beea53ef64598f4ab3 WatchSource:0}: Error finding container b0f16c3391419f1edb8a37f7ecc0299452bbbe114826d4beea53ef64598f4ab3: Status 404 returned error can't find the container with id b0f16c3391419f1edb8a37f7ecc0299452bbbe114826d4beea53ef64598f4ab3 Jul 15 19:01:39.376618 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:39.376577 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z65bn" event={"ID":"9de406e7-cfc2-4a52-b55b-c7819d750e2a","Type":"ContainerStarted","Data":"b0f16c3391419f1edb8a37f7ecc0299452bbbe114826d4beea53ef64598f4ab3"} Jul 15 19:01:39.378578 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:39.378550 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" event={"ID":"361939d2-4fef-45ba-9488-1b42cc04c7a9","Type":"ContainerStarted","Data":"a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457"} Jul 15 19:01:39.378698 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:39.378598 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" event={"ID":"361939d2-4fef-45ba-9488-1b42cc04c7a9","Type":"ContainerStarted","Data":"0fd43a8973b43e6cb71f0d3f268df64508b2fda5f78d770bd06d8f7f0c61ce43"} Jul 15 19:01:39.378775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:39.378691 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:01:39.398597 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:39.398548 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" podStartSLOduration=67.398528779 podStartE2EDuration="1m7.398528779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:39.397935762 +0000 UTC m=+68.864910075" watchObservedRunningTime="2026-07-15 19:01:39.398528779 +0000 UTC m=+68.865503093" Jul 15 19:01:41.387671 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.387640 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z65bn" event={"ID":"9de406e7-cfc2-4a52-b55b-c7819d750e2a","Type":"ContainerStarted","Data":"c33f38e7f50b41aa4e5fab76d576e72838f5a1320041141c575e7bccbead16ce"} Jul 15 19:01:41.388046 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.387696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z65bn" event={"ID":"9de406e7-cfc2-4a52-b55b-c7819d750e2a","Type":"ContainerStarted","Data":"626e25c2f7947ac622be7ae16f133a3e8ea8f7381c47c38570121804f6654450"} Jul 15 19:01:41.388046 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.387790 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:41.406477 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.406421 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z65bn" podStartSLOduration=33.380101046 podStartE2EDuration="35.406405536s" totalImagesPullingTime="2.02630449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:41.405789064 +0000 UTC m=+70.872763376" watchObservedRunningTime="2026-07-15 19:01:41.406405536 +0000 UTC m=+70.873379846" Jul 15 19:01:41.965340 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.965306 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:41.965577 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.965373 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:41.965705 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:41.965692 2572 scope.go:117] "RemoveContainer" containerID="71b44f40ad1ed184c379ca0450ff1d166ec6a66126758d96fdd0a79ae23b1d57" Jul 15 19:01:41.965875 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:41.965855 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-2kz84_openshift-console-operator(cf3ec3be-080b-4691-838d-2a72b6e7192e)\"" pod="openshift-console-operator/console-operator-7866497494-2kz84" podUID="cf3ec3be-080b-4691-838d-2a72b6e7192e" Jul 15 19:01:43.497936 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.497891 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:43.500513 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.500480 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/613706c9-971d-4afb-bec1-093b211a3b51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wgxbr\" (UID: \"613706c9-971d-4afb-bec1-093b211a3b51\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:43.598567 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.598522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:43.598757 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.598605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:43.598757 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.598657 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:43.599418 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.599394 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b656aa6d-2d76-41d4-a398-cc3bfc725624-service-ca-bundle\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:43.601462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.601434 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b656aa6d-2d76-41d4-a398-cc3bfc725624-metrics-certs\") pod \"router-default-6dffcc884d-qt2hp\" (UID: \"b656aa6d-2d76-41d4-a398-cc3bfc725624\") " pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:43.601573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.601493 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6b8b1082-bbcf-43e2-9af8-3f9575c46c48-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dv4wf\" (UID: \"6b8b1082-bbcf-43e2-9af8-3f9575c46c48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:43.632385 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.632353 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4fh54\"" Jul 15 19:01:43.639999 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.639970 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" Jul 15 19:01:43.750085 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.750006 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-699qz\"" Jul 15 19:01:43.757424 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.757404 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" Jul 15 19:01:43.794246 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.794205 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xxz89\"" Jul 15 19:01:43.801813 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:43.801790 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:44.739026 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:44.738996 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dffcc884d-qt2hp"] Jul 15 19:01:44.742461 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:44.742428 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb656aa6d_2d76_41d4_a398_cc3bfc725624.slice/crio-b46f7147d32be8c76c5fecc9e5c9802f7cbed844a073ac12d4df5fb1f48c02bb WatchSource:0}: Error finding container b46f7147d32be8c76c5fecc9e5c9802f7cbed844a073ac12d4df5fb1f48c02bb: Status 404 returned error can't find the container with id b46f7147d32be8c76c5fecc9e5c9802f7cbed844a073ac12d4df5fb1f48c02bb Jul 15 19:01:44.957792 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:44.957760 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr"] Jul 15 19:01:44.962355 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:44.962136 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf"] Jul 15 19:01:44.962355 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:44.962155 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613706c9_971d_4afb_bec1_093b211a3b51.slice/crio-eca8d49bb1ae1e565f5d9b732226ef1728307b6df3de66eb9f8c2bd3def848bc WatchSource:0}: Error finding container eca8d49bb1ae1e565f5d9b732226ef1728307b6df3de66eb9f8c2bd3def848bc: Status 404 returned error can't find the container with id eca8d49bb1ae1e565f5d9b732226ef1728307b6df3de66eb9f8c2bd3def848bc Jul 15 19:01:45.406103 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.406070 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" event={"ID":"b656aa6d-2d76-41d4-a398-cc3bfc725624","Type":"ContainerStarted","Data":"81fe4918ac94761d84c2ba250e0d379bd72212c516c6207307bb40c24c85d3c2"} Jul 15 19:01:45.406278 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.406126 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" event={"ID":"b656aa6d-2d76-41d4-a398-cc3bfc725624","Type":"ContainerStarted","Data":"b46f7147d32be8c76c5fecc9e5c9802f7cbed844a073ac12d4df5fb1f48c02bb"} Jul 15 19:01:45.407530 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.407505 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b4dg5" event={"ID":"35b4663c-ea09-4be7-a31c-660e76c46799","Type":"ContainerStarted","Data":"32b052416d575a162297d5a5ad9c51913e8f882b7bba4c5e128ca3f55ed4c620"} Jul 15 19:01:45.408610 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.408583 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" event={"ID":"613706c9-971d-4afb-bec1-093b211a3b51","Type":"ContainerStarted","Data":"eca8d49bb1ae1e565f5d9b732226ef1728307b6df3de66eb9f8c2bd3def848bc"} Jul 15 19:01:45.409694 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.409674 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" event={"ID":"6b8b1082-bbcf-43e2-9af8-3f9575c46c48","Type":"ContainerStarted","Data":"44bdb1eaf6e796543375c0e6696083e824df20e71ebe1ae5edf1df647784c424"} Jul 15 19:01:45.426272 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.426205 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" podStartSLOduration=34.426193574 podStartE2EDuration="34.426193574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:45.425503331 +0000 UTC m=+74.892477645" watchObservedRunningTime="2026-07-15 19:01:45.426193574 +0000 UTC m=+74.893167863" Jul 15 19:01:45.441474 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.441439 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b4dg5" podStartSLOduration=65.081901697 podStartE2EDuration="1m13.4414268s" totalImagesPullingTime="8.359525103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:45.441327022 +0000 UTC m=+74.908301340" watchObservedRunningTime="2026-07-15 19:01:45.4414268 +0000 UTC m=+74.908401111" Jul 15 19:01:45.803075 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.802993 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:45.805608 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:45.805587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:46.412922 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:46.412880 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:46.414040 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:46.414018 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6dffcc884d-qt2hp" Jul 15 19:01:48.421413 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:48.421379 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" event={"ID":"6b8b1082-bbcf-43e2-9af8-3f9575c46c48","Type":"ContainerStarted","Data":"80f2acdb7e210afaac41c0f92d2d5cbe7f49af23c24fb2eeed0f26d273d9d483"} Jul 15 19:01:48.421764 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:48.421446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" event={"ID":"6b8b1082-bbcf-43e2-9af8-3f9575c46c48","Type":"ContainerStarted","Data":"21f6855001dd30114bd92b56de55b7d37b1203046bfee0772f6d313bac20379e"} Jul 15 19:01:48.440738 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:48.440689 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dv4wf" podStartSLOduration=34.415972879 podStartE2EDuration="37.440673038s" totalImagesPullingTime="3.024700159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:48.438914235 +0000 UTC m=+77.905888546" watchObservedRunningTime="2026-07-15 19:01:48.440673038 +0000 UTC m=+77.907647348" Jul 15 19:01:49.426067 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:49.426025 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" event={"ID":"613706c9-971d-4afb-bec1-093b211a3b51","Type":"ContainerStarted","Data":"abad48147df948c59a1c160a2bd4e3359dac721a095eb2d0e2073a1ac20e92f3"} Jul 15 19:01:49.443943 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:49.443897 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wgxbr" podStartSLOduration=35.017001178 podStartE2EDuration="38.443881622s" totalImagesPullingTime="3.426880444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:49.443380716 +0000 UTC m=+78.910355028" watchObservedRunningTime="2026-07-15 19:01:49.443881622 +0000 UTC m=+78.910855930" Jul 15 19:01:50.763198 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:50.763162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:50.765551 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:50.765529 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/183b0d00-adc2-4d7a-806b-916a026874f1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlj2j\" (UID: \"183b0d00-adc2-4d7a-806b-916a026874f1\") " pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:51.038460 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:51.038379 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tn2w9\"" Jul 15 19:01:51.045967 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:51.045949 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xlj2j" Jul 15 19:01:51.174059 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:51.174032 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xlj2j"] Jul 15 19:01:51.181344 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:51.181303 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183b0d00_adc2_4d7a_806b_916a026874f1.slice/crio-3ff0a3c3374660d8bd57a9a2b57b2df2e56a81669f9b088d29aa595035623ed6 WatchSource:0}: Error finding container 3ff0a3c3374660d8bd57a9a2b57b2df2e56a81669f9b088d29aa595035623ed6: Status 404 returned error can't find the container with id 3ff0a3c3374660d8bd57a9a2b57b2df2e56a81669f9b088d29aa595035623ed6 Jul 15 19:01:51.397534 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:51.397499 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z65bn" Jul 15 19:01:51.434343 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:51.434310 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlj2j" event={"ID":"183b0d00-adc2-4d7a-806b-916a026874f1","Type":"ContainerStarted","Data":"3ff0a3c3374660d8bd57a9a2b57b2df2e56a81669f9b088d29aa595035623ed6"} Jul 15 19:01:53.526810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.526692 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs"] Jul 15 19:01:53.528971 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.528949 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:53.532044 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:53.532009 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="b7314e11-0169-428d-94a0-d88a8a1da34f" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:53.532194 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:53.532105 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" type="*v1.Secret" Jul 15 19:01:53.532194 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:53.532164 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 15 19:01:53.532367 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:53.532336 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" Jul 15 19:01:53.532614 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:53.532589 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 15 19:01:53.533762 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.533744 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l78mj\"" Jul 15 19:01:53.549951 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.549926 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs"] Jul 15 19:01:53.586363 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.586333 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7314e11-0169-428d-94a0-d88a8a1da34f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:53.586470 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.586390 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsxkf\" (UniqueName: \"kubernetes.io/projected/b7314e11-0169-428d-94a0-d88a8a1da34f-kube-api-access-hsxkf\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:53.611822 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.611787 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs"] Jul 15 19:01:53.613879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.613860 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:01:53.616480 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.616462 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 15 19:01:53.616784 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.616770 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rsc52\"" Jul 15 19:01:53.627711 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.627688 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs"] Jul 15 19:01:53.687057 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.687030 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40b05f0f-c537-49f0-99f8-c51506a2a09a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-9pnfs\" (UID: \"40b05f0f-c537-49f0-99f8-c51506a2a09a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:01:53.687195 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.687086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7314e11-0169-428d-94a0-d88a8a1da34f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:53.687195 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.687145 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsxkf\" (UniqueName: \"kubernetes.io/projected/b7314e11-0169-428d-94a0-d88a8a1da34f-kube-api-access-hsxkf\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:53.787792 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.787708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40b05f0f-c537-49f0-99f8-c51506a2a09a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-9pnfs\" (UID: \"40b05f0f-c537-49f0-99f8-c51506a2a09a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:01:53.790124 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.790095 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40b05f0f-c537-49f0-99f8-c51506a2a09a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-9pnfs\" (UID: \"40b05f0f-c537-49f0-99f8-c51506a2a09a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:01:53.922861 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:53.922823 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:01:54.049126 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:54.049039 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs"] Jul 15 19:01:54.051927 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:54.051898 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b05f0f_c537_49f0_99f8_c51506a2a09a.slice/crio-5801bb0b8381d1daec2804b1113e65ccb1639a7d38c9364f9e1e2b6de6612c41 WatchSource:0}: Error finding container 5801bb0b8381d1daec2804b1113e65ccb1639a7d38c9364f9e1e2b6de6612c41: Status 404 returned error can't find the container with id 5801bb0b8381d1daec2804b1113e65ccb1639a7d38c9364f9e1e2b6de6612c41 Jul 15 19:01:54.445754 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:54.445714 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" event={"ID":"40b05f0f-c537-49f0-99f8-c51506a2a09a","Type":"ContainerStarted","Data":"5801bb0b8381d1daec2804b1113e65ccb1639a7d38c9364f9e1e2b6de6612c41"} Jul 15 19:01:54.688734 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:54.688702 2572 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/managed-serviceaccount-hub-kubeconfig: failed to sync secret cache: timed out waiting for the condition Jul 15 19:01:54.689123 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:54.688802 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7314e11-0169-428d-94a0-d88a8a1da34f-hub-kubeconfig podName:b7314e11-0169-428d-94a0-d88a8a1da34f nodeName:}" failed. No retries permitted until 2026-07-15 19:01:55.188778625 +0000 UTC m=+84.655752934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "hub-kubeconfig" (UniqueName: "kubernetes.io/secret/b7314e11-0169-428d-94a0-d88a8a1da34f-hub-kubeconfig") pod "managed-serviceaccount-addon-agent-847dd87f46-sskvs" (UID: "b7314e11-0169-428d-94a0-d88a8a1da34f") : failed to sync secret cache: timed out waiting for the condition Jul 15 19:01:54.696580 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:54.696504 2572 projected.go:291] Couldn't get configMap open-cluster-management-agent-addon/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 15 19:01:54.713981 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:54.713774 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 19:01:54.741946 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:54.741916 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 19:01:54.809931 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:54.809901 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 15 19:01:54.896494 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:54.896464 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 19:01:54.896686 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:54.896609 2572 projected.go:196] Error preparing data for projected volume kube-api-access-hsxkf for pod open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs: failed to sync configmap cache: timed out waiting for the condition Jul 15 19:01:54.896748 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:01:54.896703 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7314e11-0169-428d-94a0-d88a8a1da34f-kube-api-access-hsxkf podName:b7314e11-0169-428d-94a0-d88a8a1da34f nodeName:}" failed. No retries permitted until 2026-07-15 19:01:55.396681573 +0000 UTC m=+84.863655875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hsxkf" (UniqueName: "kubernetes.io/projected/b7314e11-0169-428d-94a0-d88a8a1da34f-kube-api-access-hsxkf") pod "managed-serviceaccount-addon-agent-847dd87f46-sskvs" (UID: "b7314e11-0169-428d-94a0-d88a8a1da34f") : failed to sync configmap cache: timed out waiting for the condition Jul 15 19:01:55.198996 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:55.198957 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7314e11-0169-428d-94a0-d88a8a1da34f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:55.201888 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:55.201857 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7314e11-0169-428d-94a0-d88a8a1da34f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:55.401147 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:55.401105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsxkf\" (UniqueName: \"kubernetes.io/projected/b7314e11-0169-428d-94a0-d88a8a1da34f-kube-api-access-hsxkf\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:55.404120 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:55.404093 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsxkf\" (UniqueName: \"kubernetes.io/projected/b7314e11-0169-428d-94a0-d88a8a1da34f-kube-api-access-hsxkf\") pod \"managed-serviceaccount-addon-agent-847dd87f46-sskvs\" (UID: \"b7314e11-0169-428d-94a0-d88a8a1da34f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:55.649283 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:55.649243 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" Jul 15 19:01:55.826446 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:55.826410 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs"] Jul 15 19:01:55.829496 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:55.829467 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7314e11_0169_428d_94a0_d88a8a1da34f.slice/crio-eabe96b3823a10a3e5c36ffc98cbeff405a63320830fc686c49f0dccb79b5a53 WatchSource:0}: Error finding container eabe96b3823a10a3e5c36ffc98cbeff405a63320830fc686c49f0dccb79b5a53: Status 404 returned error can't find the container with id eabe96b3823a10a3e5c36ffc98cbeff405a63320830fc686c49f0dccb79b5a53 Jul 15 19:01:56.028004 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:56.027934 2572 scope.go:117] "RemoveContainer" containerID="71b44f40ad1ed184c379ca0450ff1d166ec6a66126758d96fdd0a79ae23b1d57" Jul 15 19:01:56.453803 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:56.453766 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" event={"ID":"b7314e11-0169-428d-94a0-d88a8a1da34f","Type":"ContainerStarted","Data":"eabe96b3823a10a3e5c36ffc98cbeff405a63320830fc686c49f0dccb79b5a53"} Jul 15 19:01:56.455560 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:56.455538 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:01:56.455683 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:56.455612 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-2kz84" event={"ID":"cf3ec3be-080b-4691-838d-2a72b6e7192e","Type":"ContainerStarted","Data":"a41c5e460b37b82f85120dc7db346c269a8778e419767af2a978623ab487a968"} Jul 15 19:01:56.455924 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:56.455901 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:56.473228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:56.473174 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-2kz84" podStartSLOduration=40.723417745 podStartE2EDuration="45.47315404s" totalImagesPullingTime="4.749736295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:56.47240572 +0000 UTC m=+85.939380057" watchObservedRunningTime="2026-07-15 19:01:56.47315404 +0000 UTC m=+85.940128354" Jul 15 19:01:57.012346 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.012315 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-2kz84" Jul 15 19:01:57.193601 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.193569 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-tvm92"] Jul 15 19:01:57.307795 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.307709 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-tvm92"] Jul 15 19:01:57.307947 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.307819 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:01:57.310490 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.310447 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 19:01:57.310631 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.310539 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 19:01:57.310631 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.310612 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tdztq\"" Jul 15 19:01:57.417825 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.417786 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv5rw\" (UniqueName: \"kubernetes.io/projected/de4044c8-ecfb-4a1f-b543-e2bca312be46-kube-api-access-jv5rw\") pod \"downloads-68c9cbb497-tvm92\" (UID: \"de4044c8-ecfb-4a1f-b543-e2bca312be46\") " pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:01:57.519582 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.519413 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv5rw\" (UniqueName: \"kubernetes.io/projected/de4044c8-ecfb-4a1f-b543-e2bca312be46-kube-api-access-jv5rw\") pod \"downloads-68c9cbb497-tvm92\" (UID: \"de4044c8-ecfb-4a1f-b543-e2bca312be46\") " pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:01:57.529130 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.529085 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv5rw\" (UniqueName: \"kubernetes.io/projected/de4044c8-ecfb-4a1f-b543-e2bca312be46-kube-api-access-jv5rw\") pod \"downloads-68c9cbb497-tvm92\" (UID: \"de4044c8-ecfb-4a1f-b543-e2bca312be46\") " pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:01:57.617516 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:57.617482 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:01:58.513329 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:58.513284 2572 patch_prober.go:28] interesting pod/image-registry-5b5d67c5b9-sjnnt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 19:01:58.513750 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:58.513352 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" podUID="361939d2-4fef-45ba-9488-1b42cc04c7a9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 19:01:59.360529 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:01:59.360496 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-tvm92"] Jul 15 19:01:59.507078 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:01:59.507008 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4044c8_ecfb_4a1f_b543_e2bca312be46.slice/crio-133712618ca04ac8d8aea4d06e015f3b1eb96893e6adac510b912e3a4347c22a WatchSource:0}: Error finding container 133712618ca04ac8d8aea4d06e015f3b1eb96893e6adac510b912e3a4347c22a: Status 404 returned error can't find the container with id 133712618ca04ac8d8aea4d06e015f3b1eb96893e6adac510b912e3a4347c22a Jul 15 19:02:00.387176 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:00.387145 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:02:00.476348 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:00.476253 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-tvm92" event={"ID":"de4044c8-ecfb-4a1f-b543-e2bca312be46","Type":"ContainerStarted","Data":"133712618ca04ac8d8aea4d06e015f3b1eb96893e6adac510b912e3a4347c22a"} Jul 15 19:02:01.482429 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.482391 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlj2j" event={"ID":"183b0d00-adc2-4d7a-806b-916a026874f1","Type":"ContainerStarted","Data":"ac3b97b60856fcc118426f243a1f8d26214454719f19823b272d583efe8caf44"} Jul 15 19:02:01.484081 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.484046 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" event={"ID":"40b05f0f-c537-49f0-99f8-c51506a2a09a","Type":"ContainerStarted","Data":"570a3b88f8ece705cff5f000342db04cfb4fe6f05e305830dd3b8fc7665a037a"} Jul 15 19:02:01.484263 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.484226 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:02:01.485979 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.485951 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" event={"ID":"b7314e11-0169-428d-94a0-d88a8a1da34f","Type":"ContainerStarted","Data":"65fffab865b7c1b69f27f3406b8b2a795c46acd06cfdec76592b37b181e9a761"} Jul 15 19:02:01.489746 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.489723 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" Jul 15 19:02:01.504658 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.504603 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9pnfs" podStartSLOduration=1.9729410600000001 podStartE2EDuration="8.504585588s" totalImagesPullingTime="6.531644528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:01.503036216 +0000 UTC m=+90.970010528" watchObservedRunningTime="2026-07-15 19:02:01.504585588 +0000 UTC m=+90.971559900" Jul 15 19:02:01.556084 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:01.556027 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-847dd87f46-sskvs" podStartSLOduration=3.735086105 podStartE2EDuration="8.556009216s" totalImagesPullingTime="4.820923111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:01.525891185 +0000 UTC m=+90.992865496" watchObservedRunningTime="2026-07-15 19:02:01.556009216 +0000 UTC m=+91.022983526" Jul 15 19:02:07.372341 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:07.372307 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2bmgw" Jul 15 19:02:08.525725 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:08.525685 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlj2j" event={"ID":"183b0d00-adc2-4d7a-806b-916a026874f1","Type":"ContainerStarted","Data":"84c85b81a35cd1f04d2602ebaffb1c3981632abb75d6ed09f93ac7a488a27852"} Jul 15 19:02:13.394326 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.394294 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bkxdv"] Jul 15 19:02:13.400775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.400749 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.403648 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.403620 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5shq7\"" Jul 15 19:02:13.403783 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.403670 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 19:02:13.403783 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.403689 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 19:02:13.403783 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.403728 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 19:02:13.403953 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.403806 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 19:02:13.466526 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.466488 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-sys\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.466706 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.466546 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.466807 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.466593 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-tls\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.467123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.467086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj7pg\" (UniqueName: \"kubernetes.io/projected/e8191ae3-b823-45c7-a04a-129e72a1b0d5-kube-api-access-sj7pg\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.467255 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.467156 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-root\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.467255 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.467198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.467374 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.467282 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8191ae3-b823-45c7-a04a-129e72a1b0d5-metrics-client-ca\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.473114 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.469039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-textfile\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.473114 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.469113 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-wtmp\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.569923 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.569883 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-tls\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.569923 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.569929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj7pg\" (UniqueName: \"kubernetes.io/projected/e8191ae3-b823-45c7-a04a-129e72a1b0d5-kube-api-access-sj7pg\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.569959 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-root\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.569991 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8191ae3-b823-45c7-a04a-129e72a1b0d5-metrics-client-ca\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570063 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-textfile\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-wtmp\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-sys\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570505 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570164 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570505 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570466 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-root\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570836 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570810 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-sys\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-wtmp\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.570941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570880 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8191ae3-b823-45c7-a04a-129e72a1b0d5-metrics-client-ca\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.571060 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.570989 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-textfile\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.571060 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.571038 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.573013 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.572992 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-tls\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.573120 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.573058 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8191ae3-b823-45c7-a04a-129e72a1b0d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.579524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.579489 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj7pg\" (UniqueName: \"kubernetes.io/projected/e8191ae3-b823-45c7-a04a-129e72a1b0d5-kube-api-access-sj7pg\") pod \"node-exporter-bkxdv\" (UID: \"e8191ae3-b823-45c7-a04a-129e72a1b0d5\") " pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:13.711477 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:13.711398 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bkxdv" Jul 15 19:02:15.551906 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:15.551871 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5d67c5b9-sjnnt"] Jul 15 19:02:19.077305 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:02:19.077270 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8191ae3_b823_45c7_a04a_129e72a1b0d5.slice/crio-ed30bc56c7c57d0472aa3f94a4df386978f084b7d861ecbdbaa67a43080e451f WatchSource:0}: Error finding container ed30bc56c7c57d0472aa3f94a4df386978f084b7d861ecbdbaa67a43080e451f: Status 404 returned error can't find the container with id ed30bc56c7c57d0472aa3f94a4df386978f084b7d861ecbdbaa67a43080e451f Jul 15 19:02:19.575737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.575680 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkxdv" event={"ID":"e8191ae3-b823-45c7-a04a-129e72a1b0d5","Type":"ContainerStarted","Data":"ed30bc56c7c57d0472aa3f94a4df386978f084b7d861ecbdbaa67a43080e451f"} Jul 15 19:02:19.578802 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.578753 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlj2j" event={"ID":"183b0d00-adc2-4d7a-806b-916a026874f1","Type":"ContainerStarted","Data":"faeca744d4d857146b81843329d737ed562be276c50aa7040c47eaac1a469383"} Jul 15 19:02:19.580760 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.580704 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-tvm92" event={"ID":"de4044c8-ecfb-4a1f-b543-e2bca312be46","Type":"ContainerStarted","Data":"abb78a7b0b6b7c0f3654674aba318dae75addab3dbf248640ea5a545efce2cf5"} Jul 15 19:02:19.581295 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.581220 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:02:19.594847 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.594759 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-tvm92" Jul 15 19:02:19.601596 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.601535 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xlj2j" podStartSLOduration=33.844848731 podStartE2EDuration="1m1.60151443s" totalImagesPullingTime="27.756665699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:01:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:19.599741904 +0000 UTC m=+109.066716212" watchObservedRunningTime="2026-07-15 19:02:19.60151443 +0000 UTC m=+109.068488745" Jul 15 19:02:19.627896 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:19.627687 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-tvm92" podStartSLOduration=3.012295863 podStartE2EDuration="22.627665691s" totalImagesPullingTime="19.615369828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:19.627014982 +0000 UTC m=+109.093989295" watchObservedRunningTime="2026-07-15 19:02:19.627665691 +0000 UTC m=+109.094640003" Jul 15 19:02:20.587595 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.587492 2572 generic.go:356] "Generic (PLEG): container finished" podID="e8191ae3-b823-45c7-a04a-129e72a1b0d5" containerID="e6bd272948c2a27d3027b0e3ffd3506b0c6dde84a7550f762447408dbe92ace7" exitCode=0 Jul 15 19:02:20.587965 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.587728 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkxdv" event={"ID":"e8191ae3-b823-45c7-a04a-129e72a1b0d5","Type":"ContainerDied","Data":"e6bd272948c2a27d3027b0e3ffd3506b0c6dde84a7550f762447408dbe92ace7"} Jul 15 19:02:20.762069 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.762016 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79c4f47487-r9k7l"] Jul 15 19:02:20.789432 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.789387 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c4f47487-r9k7l"] Jul 15 19:02:20.789594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.789542 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.792708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.792681 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 19:02:20.792873 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.792730 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 19:02:20.792985 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.792967 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5mhbd\"" Jul 15 19:02:20.793200 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.793182 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 19:02:20.793492 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.793468 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 19:02:20.793590 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.793571 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 19:02:20.798366 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.798344 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 19:02:20.939450 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939408 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-serving-cert\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.939450 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939456 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-trusted-ca-bundle\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.939687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939494 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjbs7\" (UniqueName: \"kubernetes.io/projected/198a6836-5d57-488a-8354-6e8eb1d2f828-kube-api-access-sjbs7\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.939687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-service-ca\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.939687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-console-config\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.939819 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939694 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-oauth-config\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:20.939819 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:20.939750 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-oauth-serving-cert\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.040994 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.040947 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-oauth-serving-cert\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.040994 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041004 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-serving-cert\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.041277 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-trusted-ca-bundle\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.041277 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjbs7\" (UniqueName: \"kubernetes.io/projected/198a6836-5d57-488a-8354-6e8eb1d2f828-kube-api-access-sjbs7\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.041277 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041100 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-service-ca\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.041277 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041129 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-console-config\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.041277 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041226 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-oauth-config\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.042003 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.041974 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-oauth-serving-cert\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.042365 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.042339 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-trusted-ca-bundle\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.042691 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.042670 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-console-config\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.042987 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.042970 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-service-ca\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.044312 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.044285 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-serving-cert\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.044404 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.044289 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-oauth-config\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.051394 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.051335 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjbs7\" (UniqueName: \"kubernetes.io/projected/198a6836-5d57-488a-8354-6e8eb1d2f828-kube-api-access-sjbs7\") pod \"console-79c4f47487-r9k7l\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.125246 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.125125 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:21.333590 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.333551 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c4f47487-r9k7l"] Jul 15 19:02:21.337977 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:02:21.337941 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198a6836_5d57_488a_8354_6e8eb1d2f828.slice/crio-b26d0d2d8840fc843b675e0116a5d568d0ff786dbead14fc5f53affdfabe4383 WatchSource:0}: Error finding container b26d0d2d8840fc843b675e0116a5d568d0ff786dbead14fc5f53affdfabe4383: Status 404 returned error can't find the container with id b26d0d2d8840fc843b675e0116a5d568d0ff786dbead14fc5f53affdfabe4383 Jul 15 19:02:21.595288 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.595184 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkxdv" event={"ID":"e8191ae3-b823-45c7-a04a-129e72a1b0d5","Type":"ContainerStarted","Data":"c77d310094131c6c53109ad5f159f4a1fe26ff7868b3d55e9681930293126bc1"} Jul 15 19:02:21.595735 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.595302 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkxdv" event={"ID":"e8191ae3-b823-45c7-a04a-129e72a1b0d5","Type":"ContainerStarted","Data":"f72703f04b9ece9d23332ef86552b60386f3b27d41ef7047b3fbad5c16c9453c"} Jul 15 19:02:21.596802 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.596752 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c4f47487-r9k7l" event={"ID":"198a6836-5d57-488a-8354-6e8eb1d2f828","Type":"ContainerStarted","Data":"b26d0d2d8840fc843b675e0116a5d568d0ff786dbead14fc5f53affdfabe4383"} Jul 15 19:02:21.622612 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:21.622558 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bkxdv" podStartSLOduration=7.491673853 podStartE2EDuration="8.622541893s" totalImagesPullingTime="1.004397362s" totalInitContainerRuntime="126.470678ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:21.620659311 +0000 UTC m=+111.087633636" watchObservedRunningTime="2026-07-15 19:02:21.622541893 +0000 UTC m=+111.089516203" Jul 15 19:02:25.019909 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.019876 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c4f47487-r9k7l"] Jul 15 19:02:25.051098 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.051069 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84f57765f7-2w4cz"] Jul 15 19:02:25.068416 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.068328 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f57765f7-2w4cz"] Jul 15 19:02:25.068642 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.068441 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186395 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186362 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-oauth-config\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186521 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186456 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxq4x\" (UniqueName: \"kubernetes.io/projected/d82fa783-386c-4ee4-9b40-3e19f2f989d2-kube-api-access-gxq4x\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186521 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186488 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-config\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186633 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186524 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-serving-cert\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186687 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186668 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-oauth-serving-cert\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186711 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-trusted-ca-bundle\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.186807 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.186782 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-service-ca\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287321 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-oauth-serving-cert\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287321 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-trusted-ca-bundle\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-service-ca\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-oauth-config\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287423 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxq4x\" (UniqueName: \"kubernetes.io/projected/d82fa783-386c-4ee4-9b40-3e19f2f989d2-kube-api-access-gxq4x\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287450 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-config\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.287558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287486 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-serving-cert\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.288028 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.287998 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-oauth-serving-cert\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.288285 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.288257 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-trusted-ca-bundle\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.289027 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.288594 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-service-ca\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.289129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.289072 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-config\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.290818 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.290793 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-serving-cert\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.291491 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.291468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-oauth-config\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.297681 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.297656 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxq4x\" (UniqueName: \"kubernetes.io/projected/d82fa783-386c-4ee4-9b40-3e19f2f989d2-kube-api-access-gxq4x\") pod \"console-84f57765f7-2w4cz\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.380353 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.380315 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:25.553802 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.553611 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f57765f7-2w4cz"] Jul 15 19:02:25.556764 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:02:25.556732 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82fa783_386c_4ee4_9b40_3e19f2f989d2.slice/crio-3dd6017317adec1e69971c2edc28b1fede1b9e8cf00024ceece9742780518e8c WatchSource:0}: Error finding container 3dd6017317adec1e69971c2edc28b1fede1b9e8cf00024ceece9742780518e8c: Status 404 returned error can't find the container with id 3dd6017317adec1e69971c2edc28b1fede1b9e8cf00024ceece9742780518e8c Jul 15 19:02:25.618447 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.618413 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f57765f7-2w4cz" event={"ID":"d82fa783-386c-4ee4-9b40-3e19f2f989d2","Type":"ContainerStarted","Data":"3dd6017317adec1e69971c2edc28b1fede1b9e8cf00024ceece9742780518e8c"} Jul 15 19:02:25.620177 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.620145 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c4f47487-r9k7l" event={"ID":"198a6836-5d57-488a-8354-6e8eb1d2f828","Type":"ContainerStarted","Data":"36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef"} Jul 15 19:02:25.642503 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:25.642449 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79c4f47487-r9k7l" podStartSLOduration=1.8150186000000001 podStartE2EDuration="5.64242954s" totalImagesPullingTime="3.82741094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:25.640994376 +0000 UTC m=+115.107968688" watchObservedRunningTime="2026-07-15 19:02:25.64242954 +0000 UTC m=+115.109403850" Jul 15 19:02:26.626859 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:26.626817 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f57765f7-2w4cz" event={"ID":"d82fa783-386c-4ee4-9b40-3e19f2f989d2","Type":"ContainerStarted","Data":"d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341"} Jul 15 19:02:26.648413 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:26.648347 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84f57765f7-2w4cz" podStartSLOduration=1.6483312460000001 podStartE2EDuration="1.648331246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:26.645830848 +0000 UTC m=+116.112805160" watchObservedRunningTime="2026-07-15 19:02:26.648331246 +0000 UTC m=+116.115305632" Jul 15 19:02:29.641268 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:29.641215 2572 generic.go:356] "Generic (PLEG): container finished" podID="4731e8b5-29d8-461e-8918-456eac75f5a8" containerID="864fe569c4da3e402b32f17e236c080f8909f2e1d8123f369951af104b6dd171" exitCode=0 Jul 15 19:02:29.641695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:29.641294 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" event={"ID":"4731e8b5-29d8-461e-8918-456eac75f5a8","Type":"ContainerDied","Data":"864fe569c4da3e402b32f17e236c080f8909f2e1d8123f369951af104b6dd171"} Jul 15 19:02:29.641695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:29.641602 2572 scope.go:117] "RemoveContainer" containerID="864fe569c4da3e402b32f17e236c080f8909f2e1d8123f369951af104b6dd171" Jul 15 19:02:30.649616 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:30.649576 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-bfxfh" event={"ID":"4731e8b5-29d8-461e-8918-456eac75f5a8","Type":"ContainerStarted","Data":"25bbfca90bafe157a3f9d8ed6aed363f8a8fab61c8a068f881bfd9fa00171bf1"} Jul 15 19:02:31.126060 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:31.126023 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:35.381339 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:35.381296 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:35.381800 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:35.381355 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:35.386392 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:35.386364 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:35.672139 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:35.672063 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:02:38.679489 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:38.679461 2572 generic.go:356] "Generic (PLEG): container finished" podID="176a3fce-370b-48ee-b6b2-71302f9d3731" containerID="64d594819bb599ba7cff789d45b611adb8f75155e39ed1af6307746989eeaba3" exitCode=0 Jul 15 19:02:38.679810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:38.679536 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" event={"ID":"176a3fce-370b-48ee-b6b2-71302f9d3731","Type":"ContainerDied","Data":"64d594819bb599ba7cff789d45b611adb8f75155e39ed1af6307746989eeaba3"} Jul 15 19:02:38.679920 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:38.679906 2572 scope.go:117] "RemoveContainer" containerID="64d594819bb599ba7cff789d45b611adb8f75155e39ed1af6307746989eeaba3" Jul 15 19:02:39.685373 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:39.685339 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fjxtl" event={"ID":"176a3fce-370b-48ee-b6b2-71302f9d3731","Type":"ContainerStarted","Data":"cef1fedc20d4dbf4ecad6b9b800fe09c6aabbbb5a9edbe2bffb726e65638bf1f"} Jul 15 19:02:39.908213 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:39.908176 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:02:39.910535 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:39.910515 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e8c81ac-0a8b-4100-8e3b-d0472538bcb5-metrics-certs\") pod \"network-metrics-daemon-kt4sp\" (UID: \"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5\") " pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:02:40.040291 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.040211 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7hbwd\"" Jul 15 19:02:40.047647 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.047629 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4sp" Jul 15 19:02:40.192648 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.192600 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kt4sp"] Jul 15 19:02:40.195135 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:02:40.195109 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8c81ac_0a8b_4100_8e3b_d0472538bcb5.slice/crio-bab127af28da940b431d7a8779384629c9a44507a9861e3673401cc5b7b59f7d WatchSource:0}: Error finding container bab127af28da940b431d7a8779384629c9a44507a9861e3673401cc5b7b59f7d: Status 404 returned error can't find the container with id bab127af28da940b431d7a8779384629c9a44507a9861e3673401cc5b7b59f7d Jul 15 19:02:40.576414 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.576373 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" podUID="361939d2-4fef-45ba-9488-1b42cc04c7a9" containerName="registry" containerID="cri-o://a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457" gracePeriod=30 Jul 15 19:02:40.690920 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.690883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kt4sp" event={"ID":"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5","Type":"ContainerStarted","Data":"bab127af28da940b431d7a8779384629c9a44507a9861e3673401cc5b7b59f7d"} Jul 15 19:02:40.864545 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.864516 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:02:40.917520 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917477 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/361939d2-4fef-45ba-9488-1b42cc04c7a9-ca-trust-extracted\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917540 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-bound-sa-token\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917579 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917612 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-certificates\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917640 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-trusted-ca\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917862 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917686 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-image-registry-private-configuration\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917862 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917717 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-installation-pull-secrets\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.917862 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.917767 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w7tb\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-kube-api-access-4w7tb\") pod \"361939d2-4fef-45ba-9488-1b42cc04c7a9\" (UID: \"361939d2-4fef-45ba-9488-1b42cc04c7a9\") " Jul 15 19:02:40.918547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.918054 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:02:40.918547 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.918072 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:02:40.920844 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.920814 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:02:40.921307 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.921278 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:02:40.921426 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.921305 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:02:40.921426 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.921375 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:02:40.921620 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.921571 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-kube-api-access-4w7tb" (OuterVolumeSpecName: "kube-api-access-4w7tb") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "kube-api-access-4w7tb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:02:40.929775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:40.929739 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/361939d2-4fef-45ba-9488-1b42cc04c7a9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "361939d2-4fef-45ba-9488-1b42cc04c7a9" (UID: "361939d2-4fef-45ba-9488-1b42cc04c7a9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:02:41.018382 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018348 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w7tb\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-kube-api-access-4w7tb\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018382 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018375 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/361939d2-4fef-45ba-9488-1b42cc04c7a9-ca-trust-extracted\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018382 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018385 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-bound-sa-token\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018394 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018403 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-registry-certificates\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018411 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/361939d2-4fef-45ba-9488-1b42cc04c7a9-trusted-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018421 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-image-registry-private-configuration\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.018594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.018431 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/361939d2-4fef-45ba-9488-1b42cc04c7a9-installation-pull-secrets\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:41.696279 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.696224 2572 generic.go:356] "Generic (PLEG): container finished" podID="361939d2-4fef-45ba-9488-1b42cc04c7a9" containerID="a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457" exitCode=0 Jul 15 19:02:41.696725 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.696283 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" event={"ID":"361939d2-4fef-45ba-9488-1b42cc04c7a9","Type":"ContainerDied","Data":"a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457"} Jul 15 19:02:41.696725 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.696305 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" Jul 15 19:02:41.696725 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.696351 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5d67c5b9-sjnnt" event={"ID":"361939d2-4fef-45ba-9488-1b42cc04c7a9","Type":"ContainerDied","Data":"0fd43a8973b43e6cb71f0d3f268df64508b2fda5f78d770bd06d8f7f0c61ce43"} Jul 15 19:02:41.696725 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.696373 2572 scope.go:117] "RemoveContainer" containerID="a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457" Jul 15 19:02:41.710389 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.710373 2572 scope.go:117] "RemoveContainer" containerID="a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457" Jul 15 19:02:41.710653 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:02:41.710630 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457\": container with ID starting with a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457 not found: ID does not exist" containerID="a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457" Jul 15 19:02:41.710719 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.710661 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457"} err="failed to get container status \"a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457\": rpc error: code = NotFound desc = could not find container \"a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457\": container with ID starting with a3066f337b593a18fa56a644a7831e95f8ec7617a17e104a096a7e121a903457 not found: ID does not exist" Jul 15 19:02:41.717889 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.717863 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5d67c5b9-sjnnt"] Jul 15 19:02:41.728558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:41.728535 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b5d67c5b9-sjnnt"] Jul 15 19:02:42.702886 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:42.702850 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kt4sp" event={"ID":"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5","Type":"ContainerStarted","Data":"32bbd15e5a51684b36bfdbe753b3bbf483d6645d6e19cc019a0454c767955288"} Jul 15 19:02:42.703387 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:42.702902 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kt4sp" event={"ID":"8e8c81ac-0a8b-4100-8e3b-d0472538bcb5","Type":"ContainerStarted","Data":"e0251006d937066253d145f2da2fb7766256a7b1c1a2407da52cc06dcd9dc671"} Jul 15 19:02:42.724445 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:42.724394 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kt4sp" podStartSLOduration=129.279572909 podStartE2EDuration="2m10.724379276s" totalImagesPullingTime="1.444806367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:02:42.722924967 +0000 UTC m=+132.189899277" watchObservedRunningTime="2026-07-15 19:02:42.724379276 +0000 UTC m=+132.191353639" Jul 15 19:02:43.032285 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:43.032180 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="361939d2-4fef-45ba-9488-1b42cc04c7a9" path="/var/lib/kubelet/pods/361939d2-4fef-45ba-9488-1b42cc04c7a9/volumes" Jul 15 19:02:43.708129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:43.708088 2572 generic.go:356] "Generic (PLEG): container finished" podID="3467ed9c-093c-4b71-b519-3dffa22383eb" containerID="ba904b67b79c3910163c1cbe4ff2a9a28defee8def70e46fe94bba4f4f02f875" exitCode=0 Jul 15 19:02:43.708503 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:43.708161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" event={"ID":"3467ed9c-093c-4b71-b519-3dffa22383eb","Type":"ContainerDied","Data":"ba904b67b79c3910163c1cbe4ff2a9a28defee8def70e46fe94bba4f4f02f875"} Jul 15 19:02:43.708655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:43.708640 2572 scope.go:117] "RemoveContainer" containerID="ba904b67b79c3910163c1cbe4ff2a9a28defee8def70e46fe94bba4f4f02f875" Jul 15 19:02:44.714315 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:44.714275 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hlmwm" event={"ID":"3467ed9c-093c-4b71-b519-3dffa22383eb","Type":"ContainerStarted","Data":"f8e9f2494d3b70c892fd22d16bbffd7245dee0a0fb2fa1553f1abd47682158bf"} Jul 15 19:02:50.645136 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:50.645085 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79c4f47487-r9k7l" podUID="198a6836-5d57-488a-8354-6e8eb1d2f828" containerName="console" containerID="cri-o://36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef" gracePeriod=15 Jul 15 19:02:50.930536 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:50.930452 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:51.007496 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007457 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-serving-cert\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.007496 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007501 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-trusted-ca-bundle\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.007729 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007534 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-oauth-serving-cert\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.007729 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007571 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-service-ca\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.007729 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007588 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-console-config\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.007729 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007605 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-oauth-config\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.007729 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007627 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjbs7\" (UniqueName: \"kubernetes.io/projected/198a6836-5d57-488a-8354-6e8eb1d2f828-kube-api-access-sjbs7\") pod \"198a6836-5d57-488a-8354-6e8eb1d2f828\" (UID: \"198a6836-5d57-488a-8354-6e8eb1d2f828\") " Jul 15 19:02:51.008007 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007972 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:02:51.008209 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.007981 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-service-ca" (OuterVolumeSpecName: "service-ca") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:02:51.008300 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.008252 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-console-config" (OuterVolumeSpecName: "console-config") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:02:51.008300 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.008281 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:02:51.009921 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.009897 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198a6836-5d57-488a-8354-6e8eb1d2f828-kube-api-access-sjbs7" (OuterVolumeSpecName: "kube-api-access-sjbs7") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "kube-api-access-sjbs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:02:51.009997 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.009903 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:02:51.009997 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.009925 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "198a6836-5d57-488a-8354-6e8eb1d2f828" (UID: "198a6836-5d57-488a-8354-6e8eb1d2f828"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:02:51.108879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108833 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-serving-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.108879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108868 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-trusted-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.108879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108878 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-oauth-serving-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.108879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108887 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.108879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108897 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/198a6836-5d57-488a-8354-6e8eb1d2f828-console-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.109270 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108904 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/198a6836-5d57-488a-8354-6e8eb1d2f828-console-oauth-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.109270 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.108915 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjbs7\" (UniqueName: \"kubernetes.io/projected/198a6836-5d57-488a-8354-6e8eb1d2f828-kube-api-access-sjbs7\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.742815 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.742775 2572 generic.go:356] "Generic (PLEG): container finished" podID="198a6836-5d57-488a-8354-6e8eb1d2f828" containerID="36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef" exitCode=0 Jul 15 19:02:51.742815 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.742818 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c4f47487-r9k7l" event={"ID":"198a6836-5d57-488a-8354-6e8eb1d2f828","Type":"ContainerDied","Data":"36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef"} Jul 15 19:02:51.743348 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.742832 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c4f47487-r9k7l" Jul 15 19:02:51.743348 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.742884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c4f47487-r9k7l" event={"ID":"198a6836-5d57-488a-8354-6e8eb1d2f828","Type":"ContainerDied","Data":"b26d0d2d8840fc843b675e0116a5d568d0ff786dbead14fc5f53affdfabe4383"} Jul 15 19:02:51.743348 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.742905 2572 scope.go:117] "RemoveContainer" containerID="36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef" Jul 15 19:02:51.754944 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.754914 2572 scope.go:117] "RemoveContainer" containerID="36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef" Jul 15 19:02:51.755228 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:02:51.755206 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef\": container with ID starting with 36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef not found: ID does not exist" containerID="36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef" Jul 15 19:02:51.755322 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.755255 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef"} err="failed to get container status \"36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef\": rpc error: code = NotFound desc = could not find container \"36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef\": container with ID starting with 36a8c29cb648987e7c3d98f5ee1b9b6138740260427a9444fd67af90915b40ef not found: ID does not exist" Jul 15 19:02:51.761819 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.761795 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c4f47487-r9k7l"] Jul 15 19:02:51.765896 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:51.765870 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79c4f47487-r9k7l"] Jul 15 19:02:53.033290 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:02:53.033253 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="198a6836-5d57-488a-8354-6e8eb1d2f828" path="/var/lib/kubelet/pods/198a6836-5d57-488a-8354-6e8eb1d2f828/volumes" Jul 15 19:03:04.267973 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.267936 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c476cc6d5-fmv6r"] Jul 15 19:03:04.268462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.268252 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a6836-5d57-488a-8354-6e8eb1d2f828" containerName="console" Jul 15 19:03:04.268462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.268265 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a6836-5d57-488a-8354-6e8eb1d2f828" containerName="console" Jul 15 19:03:04.268462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.268293 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="361939d2-4fef-45ba-9488-1b42cc04c7a9" containerName="registry" Jul 15 19:03:04.268462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.268299 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="361939d2-4fef-45ba-9488-1b42cc04c7a9" containerName="registry" Jul 15 19:03:04.268462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.268352 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="198a6836-5d57-488a-8354-6e8eb1d2f828" containerName="console" Jul 15 19:03:04.268462 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.268363 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="361939d2-4fef-45ba-9488-1b42cc04c7a9" containerName="registry" Jul 15 19:03:04.285024 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.284985 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c476cc6d5-fmv6r"] Jul 15 19:03:04.285266 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.285100 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.412721 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412685 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5jhg\" (UniqueName: \"kubernetes.io/projected/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-kube-api-access-d5jhg\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.412892 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-oauth-config\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.412892 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-oauth-serving-cert\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.412892 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412877 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-trusted-ca-bundle\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.413010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412899 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-config\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.413010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412931 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-service-ca\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.413010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.412954 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-serving-cert\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514291 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-oauth-config\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514476 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514398 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-oauth-serving-cert\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514518 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514483 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-trusted-ca-bundle\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514518 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514511 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-config\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514581 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514544 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-service-ca\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514803 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514763 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-serving-cert\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.514879 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.514834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5jhg\" (UniqueName: \"kubernetes.io/projected/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-kube-api-access-d5jhg\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.515205 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.515175 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-oauth-serving-cert\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.515360 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.515324 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-config\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.515442 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.515419 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-service-ca\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.515586 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.515571 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-trusted-ca-bundle\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.517294 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.517274 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-oauth-config\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.517422 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.517404 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-serving-cert\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.525654 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.525589 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5jhg\" (UniqueName: \"kubernetes.io/projected/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-kube-api-access-d5jhg\") pod \"console-6c476cc6d5-fmv6r\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.595595 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.595552 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:04.767977 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.767952 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c476cc6d5-fmv6r"] Jul 15 19:03:04.770578 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:03:04.770551 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc168814_5f4e_4f59_8c63_ab8c184b8ef7.slice/crio-87226c9d2f127625cd8d9da54f7ccb6861a2bfa918e68b6dcaf4c58b800d7185 WatchSource:0}: Error finding container 87226c9d2f127625cd8d9da54f7ccb6861a2bfa918e68b6dcaf4c58b800d7185: Status 404 returned error can't find the container with id 87226c9d2f127625cd8d9da54f7ccb6861a2bfa918e68b6dcaf4c58b800d7185 Jul 15 19:03:04.797423 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:04.797342 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c476cc6d5-fmv6r" event={"ID":"dc168814-5f4e-4f59-8c63-ab8c184b8ef7","Type":"ContainerStarted","Data":"87226c9d2f127625cd8d9da54f7ccb6861a2bfa918e68b6dcaf4c58b800d7185"} Jul 15 19:03:05.803952 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:05.803908 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c476cc6d5-fmv6r" event={"ID":"dc168814-5f4e-4f59-8c63-ab8c184b8ef7","Type":"ContainerStarted","Data":"9c0683e300690c56b28ac00da8ba406fb33500f74b168c6cf66909bc5a1fdd02"} Jul 15 19:03:05.838651 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:05.838596 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c476cc6d5-fmv6r" podStartSLOduration=1.8385794949999998 podStartE2EDuration="1.838579495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:03:05.837728159 +0000 UTC m=+155.304702472" watchObservedRunningTime="2026-07-15 19:03:05.838579495 +0000 UTC m=+155.305553806" Jul 15 19:03:14.596569 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:14.596522 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:14.597036 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:14.596591 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:14.601425 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:14.601401 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:14.845852 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:14.845820 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:03:14.904182 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:14.904089 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84f57765f7-2w4cz"] Jul 15 19:03:39.928841 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:39.928798 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84f57765f7-2w4cz" podUID="d82fa783-386c-4ee4-9b40-3e19f2f989d2" containerName="console" containerID="cri-o://d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341" gracePeriod=15 Jul 15 19:03:40.170828 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.170799 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:03:40.198523 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198430 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-oauth-config\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.198523 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198475 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-trusted-ca-bundle\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.198726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198522 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-config\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.198726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198576 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxq4x\" (UniqueName: \"kubernetes.io/projected/d82fa783-386c-4ee4-9b40-3e19f2f989d2-kube-api-access-gxq4x\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.198726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198601 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-serving-cert\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.198726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198629 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-oauth-serving-cert\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.198726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.198662 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-service-ca\") pod \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\" (UID: \"d82fa783-386c-4ee4-9b40-3e19f2f989d2\") " Jul 15 19:03:40.199507 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.199249 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-service-ca" (OuterVolumeSpecName: "service-ca") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:03:40.199795 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.199672 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:03:40.199795 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.199766 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-config" (OuterVolumeSpecName: "console-config") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:03:40.199934 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.199773 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:03:40.200928 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.200900 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:03:40.201533 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.201512 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:03:40.201615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.201526 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d82fa783-386c-4ee4-9b40-3e19f2f989d2-kube-api-access-gxq4x" (OuterVolumeSpecName: "kube-api-access-gxq4x") pod "d82fa783-386c-4ee4-9b40-3e19f2f989d2" (UID: "d82fa783-386c-4ee4-9b40-3e19f2f989d2"). InnerVolumeSpecName "kube-api-access-gxq4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:03:40.300132 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300095 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.300132 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300128 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxq4x\" (UniqueName: \"kubernetes.io/projected/d82fa783-386c-4ee4-9b40-3e19f2f989d2-kube-api-access-gxq4x\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.300132 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300140 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-serving-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.300396 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300149 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-oauth-serving-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.300396 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300159 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.300396 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300167 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d82fa783-386c-4ee4-9b40-3e19f2f989d2-console-oauth-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.300396 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.300175 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d82fa783-386c-4ee4-9b40-3e19f2f989d2-trusted-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:03:40.948243 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.948203 2572 generic.go:356] "Generic (PLEG): container finished" podID="d82fa783-386c-4ee4-9b40-3e19f2f989d2" containerID="d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341" exitCode=0 Jul 15 19:03:40.948636 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.948265 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f57765f7-2w4cz" Jul 15 19:03:40.948636 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.948287 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f57765f7-2w4cz" event={"ID":"d82fa783-386c-4ee4-9b40-3e19f2f989d2","Type":"ContainerDied","Data":"d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341"} Jul 15 19:03:40.948636 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.948373 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f57765f7-2w4cz" event={"ID":"d82fa783-386c-4ee4-9b40-3e19f2f989d2","Type":"ContainerDied","Data":"3dd6017317adec1e69971c2edc28b1fede1b9e8cf00024ceece9742780518e8c"} Jul 15 19:03:40.948636 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.948396 2572 scope.go:117] "RemoveContainer" containerID="d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341" Jul 15 19:03:40.957273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.957245 2572 scope.go:117] "RemoveContainer" containerID="d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341" Jul 15 19:03:40.957555 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:03:40.957536 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341\": container with ID starting with d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341 not found: ID does not exist" containerID="d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341" Jul 15 19:03:40.957611 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.957567 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341"} err="failed to get container status \"d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341\": rpc error: code = NotFound desc = could not find container \"d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341\": container with ID starting with d7df8b1d9648e71aa3eb8355187fc64f84ca43079e971883eb6870c811ff3341 not found: ID does not exist" Jul 15 19:03:40.971663 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.971627 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84f57765f7-2w4cz"] Jul 15 19:03:40.979940 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:40.979907 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84f57765f7-2w4cz"] Jul 15 19:03:41.032849 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:03:41.032817 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d82fa783-386c-4ee4-9b40-3e19f2f989d2" path="/var/lib/kubelet/pods/d82fa783-386c-4ee4-9b40-3e19f2f989d2/volumes" Jul 15 19:04:44.031705 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.031669 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7"] Jul 15 19:04:44.032167 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.031944 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d82fa783-386c-4ee4-9b40-3e19f2f989d2" containerName="console" Jul 15 19:04:44.032167 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.031954 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82fa783-386c-4ee4-9b40-3e19f2f989d2" containerName="console" Jul 15 19:04:44.032167 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.032009 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d82fa783-386c-4ee4-9b40-3e19f2f989d2" containerName="console" Jul 15 19:04:44.034520 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.034503 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.039610 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.039583 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:04:44.039610 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.039606 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ld255\"" Jul 15 19:04:44.039995 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.039976 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 19:04:44.052837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.052816 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7"] Jul 15 19:04:44.182038 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.182000 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sjh6\" (UniqueName: \"kubernetes.io/projected/428a0048-9893-4d2a-b64e-46ab7e08d511-kube-api-access-2sjh6\") pod \"cert-manager-operator-controller-manager-55c444bf86-c9wj7\" (UID: \"428a0048-9893-4d2a-b64e-46ab7e08d511\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.182214 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.182060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/428a0048-9893-4d2a-b64e-46ab7e08d511-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c9wj7\" (UID: \"428a0048-9893-4d2a-b64e-46ab7e08d511\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.283136 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.283061 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/428a0048-9893-4d2a-b64e-46ab7e08d511-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c9wj7\" (UID: \"428a0048-9893-4d2a-b64e-46ab7e08d511\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.283289 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.283143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sjh6\" (UniqueName: \"kubernetes.io/projected/428a0048-9893-4d2a-b64e-46ab7e08d511-kube-api-access-2sjh6\") pod \"cert-manager-operator-controller-manager-55c444bf86-c9wj7\" (UID: \"428a0048-9893-4d2a-b64e-46ab7e08d511\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.283459 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.283438 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/428a0048-9893-4d2a-b64e-46ab7e08d511-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c9wj7\" (UID: \"428a0048-9893-4d2a-b64e-46ab7e08d511\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.295084 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.295059 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sjh6\" (UniqueName: \"kubernetes.io/projected/428a0048-9893-4d2a-b64e-46ab7e08d511-kube-api-access-2sjh6\") pod \"cert-manager-operator-controller-manager-55c444bf86-c9wj7\" (UID: \"428a0048-9893-4d2a-b64e-46ab7e08d511\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.343971 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.343945 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" Jul 15 19:04:44.480070 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:44.480039 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7"] Jul 15 19:04:44.483577 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:04:44.483551 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod428a0048_9893_4d2a_b64e_46ab7e08d511.slice/crio-f7df7d51a186d68a00ceae3510d07e7fa73cfbe0140c165a5008cdfdb589a935 WatchSource:0}: Error finding container f7df7d51a186d68a00ceae3510d07e7fa73cfbe0140c165a5008cdfdb589a935: Status 404 returned error can't find the container with id f7df7d51a186d68a00ceae3510d07e7fa73cfbe0140c165a5008cdfdb589a935 Jul 15 19:04:45.194868 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:45.194827 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" event={"ID":"428a0048-9893-4d2a-b64e-46ab7e08d511","Type":"ContainerStarted","Data":"f7df7d51a186d68a00ceae3510d07e7fa73cfbe0140c165a5008cdfdb589a935"} Jul 15 19:04:48.209742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.209705 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" event={"ID":"428a0048-9893-4d2a-b64e-46ab7e08d511","Type":"ContainerStarted","Data":"142ed8866cdff92aa90fb30634af08d2a40f0d0810fe8f0f86ac7def45f85a7d"} Jul 15 19:04:48.233105 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.233049 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c9wj7" podStartSLOduration=1.5360933650000002 podStartE2EDuration="4.233029489s" totalImagesPullingTime="2.696936124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:04:48.231228909 +0000 UTC m=+257.698203222" watchObservedRunningTime="2026-07-15 19:04:48.233029489 +0000 UTC m=+257.700003802" Jul 15 19:04:48.933133 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.933092 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj"] Jul 15 19:04:48.935705 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.935688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:48.938621 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.938599 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:04:48.938731 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.938638 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-45lbk\"" Jul 15 19:04:48.938731 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.938608 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:04:48.948970 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:48.948936 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj"] Jul 15 19:04:49.025001 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.024966 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.025162 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.025015 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-kube-api-access-q5mzp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.025162 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.025096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.125639 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.125603 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.125639 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.125644 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-kube-api-access-q5mzp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.125840 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.125672 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.125992 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.125974 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.126028 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.126004 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.142993 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.142966 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-kube-api-access-q5mzp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.245614 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.245538 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:04:49.378744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:49.378720 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj"] Jul 15 19:04:49.381326 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:04:49.381297 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e1ce84_4638_4eb6_b9c2_6f6d5580249d.slice/crio-0d86f900b8b891b0d1a39c37f2b5df057c8825b98aa90c7f376606f5c2ac790a WatchSource:0}: Error finding container 0d86f900b8b891b0d1a39c37f2b5df057c8825b98aa90c7f376606f5c2ac790a: Status 404 returned error can't find the container with id 0d86f900b8b891b0d1a39c37f2b5df057c8825b98aa90c7f376606f5c2ac790a Jul 15 19:04:50.220012 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:50.219972 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" event={"ID":"39e1ce84-4638-4eb6-b9c2-6f6d5580249d","Type":"ContainerStarted","Data":"0d86f900b8b891b0d1a39c37f2b5df057c8825b98aa90c7f376606f5c2ac790a"} Jul 15 19:04:56.253873 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:56.253843 2572 generic.go:356] "Generic (PLEG): container finished" podID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerID="a22031d6e870234aac458fea39cd2cc4e928e6a5c60f026ce7f672559f157efc" exitCode=0 Jul 15 19:04:56.254203 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:56.253904 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" event={"ID":"39e1ce84-4638-4eb6-b9c2-6f6d5580249d","Type":"ContainerDied","Data":"a22031d6e870234aac458fea39cd2cc4e928e6a5c60f026ce7f672559f157efc"} Jul 15 19:04:58.265744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.265616 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" event={"ID":"39e1ce84-4638-4eb6-b9c2-6f6d5580249d","Type":"ContainerStarted","Data":"0517001624f3976153b18ff0fa85331343755ed2fac9a5ef772d0c095f5d48ac"} Jul 15 19:04:58.585493 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.585414 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6gbcm"] Jul 15 19:04:58.588600 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.588582 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.591551 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.591521 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 19:04:58.591671 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.591555 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n8l56\"" Jul 15 19:04:58.591671 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.591561 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 19:04:58.600379 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.600354 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6gbcm"] Jul 15 19:04:58.608475 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.608442 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4l4p\" (UniqueName: \"kubernetes.io/projected/3b9a57c9-ebd2-4660-bd23-3affcc7da219-kube-api-access-q4l4p\") pod \"cert-manager-55b4d954d5-6gbcm\" (UID: \"3b9a57c9-ebd2-4660-bd23-3affcc7da219\") " pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.608596 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.608544 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b9a57c9-ebd2-4660-bd23-3affcc7da219-bound-sa-token\") pod \"cert-manager-55b4d954d5-6gbcm\" (UID: \"3b9a57c9-ebd2-4660-bd23-3affcc7da219\") " pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.709078 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.709042 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b9a57c9-ebd2-4660-bd23-3affcc7da219-bound-sa-token\") pod \"cert-manager-55b4d954d5-6gbcm\" (UID: \"3b9a57c9-ebd2-4660-bd23-3affcc7da219\") " pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.709280 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.709120 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4l4p\" (UniqueName: \"kubernetes.io/projected/3b9a57c9-ebd2-4660-bd23-3affcc7da219-kube-api-access-q4l4p\") pod \"cert-manager-55b4d954d5-6gbcm\" (UID: \"3b9a57c9-ebd2-4660-bd23-3affcc7da219\") " pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.718762 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.718737 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b9a57c9-ebd2-4660-bd23-3affcc7da219-bound-sa-token\") pod \"cert-manager-55b4d954d5-6gbcm\" (UID: \"3b9a57c9-ebd2-4660-bd23-3affcc7da219\") " pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.718995 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.718976 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4l4p\" (UniqueName: \"kubernetes.io/projected/3b9a57c9-ebd2-4660-bd23-3affcc7da219-kube-api-access-q4l4p\") pod \"cert-manager-55b4d954d5-6gbcm\" (UID: \"3b9a57c9-ebd2-4660-bd23-3affcc7da219\") " pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:58.897535 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:58.897497 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6gbcm" Jul 15 19:04:59.025767 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:59.025727 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6gbcm"] Jul 15 19:04:59.029079 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:04:59.029052 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b9a57c9_ebd2_4660_bd23_3affcc7da219.slice/crio-764957f6337233912edc3dc24c030d8a291e7d5d75e9cf62a085a03e40992a5a WatchSource:0}: Error finding container 764957f6337233912edc3dc24c030d8a291e7d5d75e9cf62a085a03e40992a5a: Status 404 returned error can't find the container with id 764957f6337233912edc3dc24c030d8a291e7d5d75e9cf62a085a03e40992a5a Jul 15 19:04:59.271801 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:59.271703 2572 generic.go:356] "Generic (PLEG): container finished" podID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerID="0517001624f3976153b18ff0fa85331343755ed2fac9a5ef772d0c095f5d48ac" exitCode=0 Jul 15 19:04:59.271801 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:59.271765 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" event={"ID":"39e1ce84-4638-4eb6-b9c2-6f6d5580249d","Type":"ContainerDied","Data":"0517001624f3976153b18ff0fa85331343755ed2fac9a5ef772d0c095f5d48ac"} Jul 15 19:04:59.273095 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:04:59.273070 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6gbcm" event={"ID":"3b9a57c9-ebd2-4660-bd23-3affcc7da219","Type":"ContainerStarted","Data":"764957f6337233912edc3dc24c030d8a291e7d5d75e9cf62a085a03e40992a5a"} Jul 15 19:05:02.288993 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:02.288936 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6gbcm" event={"ID":"3b9a57c9-ebd2-4660-bd23-3affcc7da219","Type":"ContainerStarted","Data":"539f8f5a331b9015d6cccd614dd6fbd0a2502cb916fefb6e5d84a2f9587016ef"} Jul 15 19:05:02.308192 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:02.308143 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6gbcm" podStartSLOduration=1.7245110160000001 podStartE2EDuration="4.30812333s" totalImagesPullingTime="2.583612314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:05:02.306534881 +0000 UTC m=+271.773509193" watchObservedRunningTime="2026-07-15 19:05:02.30812333 +0000 UTC m=+271.775097640" Jul 15 19:05:07.313516 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:07.313472 2572 generic.go:356] "Generic (PLEG): container finished" podID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerID="aeb61ef91041029866fe381c304ea90f7e7d3011fad50dc4be127fc418da6e6d" exitCode=0 Jul 15 19:05:07.313888 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:07.313556 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" event={"ID":"39e1ce84-4638-4eb6-b9c2-6f6d5580249d","Type":"ContainerDied","Data":"aeb61ef91041029866fe381c304ea90f7e7d3011fad50dc4be127fc418da6e6d"} Jul 15 19:05:08.441493 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.441466 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:05:08.489150 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.489120 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-kube-api-access-q5mzp\") pod \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " Jul 15 19:05:08.489288 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.489186 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-bundle\") pod \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " Jul 15 19:05:08.489288 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.489214 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-util\") pod \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\" (UID: \"39e1ce84-4638-4eb6-b9c2-6f6d5580249d\") " Jul 15 19:05:08.489628 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.489602 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-bundle" (OuterVolumeSpecName: "bundle") pod "39e1ce84-4638-4eb6-b9c2-6f6d5580249d" (UID: "39e1ce84-4638-4eb6-b9c2-6f6d5580249d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:08.491326 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.491292 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-kube-api-access-q5mzp" (OuterVolumeSpecName: "kube-api-access-q5mzp") pod "39e1ce84-4638-4eb6-b9c2-6f6d5580249d" (UID: "39e1ce84-4638-4eb6-b9c2-6f6d5580249d"). InnerVolumeSpecName "kube-api-access-q5mzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:05:08.493741 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.493719 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-util" (OuterVolumeSpecName: "util") pod "39e1ce84-4638-4eb6-b9c2-6f6d5580249d" (UID: "39e1ce84-4638-4eb6-b9c2-6f6d5580249d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:08.590484 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.590407 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5mzp\" (UniqueName: \"kubernetes.io/projected/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-kube-api-access-q5mzp\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:05:08.590484 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.590435 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:05:08.590484 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:08.590445 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39e1ce84-4638-4eb6-b9c2-6f6d5580249d-util\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:05:09.323944 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:09.323867 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" event={"ID":"39e1ce84-4638-4eb6-b9c2-6f6d5580249d","Type":"ContainerDied","Data":"0d86f900b8b891b0d1a39c37f2b5df057c8825b98aa90c7f376606f5c2ac790a"} Jul 15 19:05:09.323944 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:09.323919 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d86f900b8b891b0d1a39c37f2b5df057c8825b98aa90c7f376606f5c2ac790a" Jul 15 19:05:09.323944 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:09.323874 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2nktj" Jul 15 19:05:30.973565 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:30.973534 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:05:30.974114 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:30.973539 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:05:30.981809 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:30.981787 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:05:30.981809 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:30.981804 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:05:30.984528 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:30.984508 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 19:05:35.309205 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309162 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl"] Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309584 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="extract" Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309603 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="extract" Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309623 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="util" Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309630 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="util" Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309642 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="pull" Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309648 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="pull" Jul 15 19:05:35.310742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.309732 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e1ce84-4638-4eb6-b9c2-6f6d5580249d" containerName="extract" Jul 15 19:05:35.311707 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.311685 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.316088 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.316066 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:05:35.316190 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.316124 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 19:05:35.317408 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.317391 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 19:05:35.317493 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.317431 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 19:05:35.317546 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.317528 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-khls9\"" Jul 15 19:05:35.319639 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.319620 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 19:05:35.383419 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.383391 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl"] Jul 15 19:05:35.399466 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.399439 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7v5\" (UniqueName: \"kubernetes.io/projected/6e59a746-0cf4-4939-8448-7ee97b78fbbf-kube-api-access-jv7v5\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.399583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.399474 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e59a746-0cf4-4939-8448-7ee97b78fbbf-metrics-cert\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.399583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.399496 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6e59a746-0cf4-4939-8448-7ee97b78fbbf-manager-config\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.399583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.399555 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e59a746-0cf4-4939-8448-7ee97b78fbbf-cert\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.500836 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.500797 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6e59a746-0cf4-4939-8448-7ee97b78fbbf-manager-config\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.501063 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.500859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e59a746-0cf4-4939-8448-7ee97b78fbbf-cert\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.501063 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.500945 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7v5\" (UniqueName: \"kubernetes.io/projected/6e59a746-0cf4-4939-8448-7ee97b78fbbf-kube-api-access-jv7v5\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.501063 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.500973 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e59a746-0cf4-4939-8448-7ee97b78fbbf-metrics-cert\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.501598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.501572 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6e59a746-0cf4-4939-8448-7ee97b78fbbf-manager-config\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.503532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.503510 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e59a746-0cf4-4939-8448-7ee97b78fbbf-cert\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.503652 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.503581 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6e59a746-0cf4-4939-8448-7ee97b78fbbf-metrics-cert\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.521628 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.521599 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7v5\" (UniqueName: \"kubernetes.io/projected/6e59a746-0cf4-4939-8448-7ee97b78fbbf-kube-api-access-jv7v5\") pod \"lws-controller-manager-8659b4c98-fdkvl\" (UID: \"6e59a746-0cf4-4939-8448-7ee97b78fbbf\") " pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.621355 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.621322 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:35.779138 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.779106 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl"] Jul 15 19:05:35.780212 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:05:35.780187 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e59a746_0cf4_4939_8448_7ee97b78fbbf.slice/crio-bd240c8854af9c2275a1def7ee3dbf15906ba182bede947889e97d8786721f87 WatchSource:0}: Error finding container bd240c8854af9c2275a1def7ee3dbf15906ba182bede947889e97d8786721f87: Status 404 returned error can't find the container with id bd240c8854af9c2275a1def7ee3dbf15906ba182bede947889e97d8786721f87 Jul 15 19:05:35.782347 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:35.782323 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:05:36.438594 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:36.438547 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" event={"ID":"6e59a746-0cf4-4939-8448-7ee97b78fbbf","Type":"ContainerStarted","Data":"bd240c8854af9c2275a1def7ee3dbf15906ba182bede947889e97d8786721f87"} Jul 15 19:05:37.450182 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:37.450080 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" event={"ID":"6e59a746-0cf4-4939-8448-7ee97b78fbbf","Type":"ContainerStarted","Data":"c883e9d9ef6cf84042acc3d0a7a787c2de81a9e0c3dc822f59cc0eb7f9cc0b13"} Jul 15 19:05:37.450587 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:37.450204 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:37.469138 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:37.469092 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" podStartSLOduration=1.113538392 podStartE2EDuration="2.46907827s" totalImagesPullingTime="1.355539878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:05:37.46892572 +0000 UTC m=+306.935900031" watchObservedRunningTime="2026-07-15 19:05:37.46907827 +0000 UTC m=+306.936052581" Jul 15 19:05:39.009330 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.009295 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd"] Jul 15 19:05:39.025988 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.025965 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.029784 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.029762 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rf8qx\"" Jul 15 19:05:39.030207 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.030178 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 19:05:39.030479 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.030461 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 19:05:39.030564 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.030490 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 19:05:39.033216 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.033193 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 19:05:39.036670 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.036650 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd"] Jul 15 19:05:39.132391 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.132354 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/736ad68d-b406-44fe-90b5-d74c095c812e-webhook-cert\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.132579 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.132400 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736ad68d-b406-44fe-90b5-d74c095c812e-tmp\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.132579 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.132528 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/736ad68d-b406-44fe-90b5-d74c095c812e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.132579 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.132566 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfmth\" (UniqueName: \"kubernetes.io/projected/736ad68d-b406-44fe-90b5-d74c095c812e-kube-api-access-dfmth\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.132729 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.132598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/736ad68d-b406-44fe-90b5-d74c095c812e-manifests\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.233712 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.233679 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/736ad68d-b406-44fe-90b5-d74c095c812e-webhook-cert\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.233894 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.233723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736ad68d-b406-44fe-90b5-d74c095c812e-tmp\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.233894 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.233765 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/736ad68d-b406-44fe-90b5-d74c095c812e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.233894 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.233783 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfmth\" (UniqueName: \"kubernetes.io/projected/736ad68d-b406-44fe-90b5-d74c095c812e-kube-api-access-dfmth\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.233894 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.233803 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/736ad68d-b406-44fe-90b5-d74c095c812e-manifests\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.234271 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.234229 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/736ad68d-b406-44fe-90b5-d74c095c812e-manifests\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.236092 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.236066 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/736ad68d-b406-44fe-90b5-d74c095c812e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.236223 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.236206 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736ad68d-b406-44fe-90b5-d74c095c812e-tmp\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.236543 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.236507 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/736ad68d-b406-44fe-90b5-d74c095c812e-webhook-cert\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.242733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.242714 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfmth\" (UniqueName: \"kubernetes.io/projected/736ad68d-b406-44fe-90b5-d74c095c812e-kube-api-access-dfmth\") pod \"opendatahub-operator-controller-manager-55c89bff45-p4cdd\" (UID: \"736ad68d-b406-44fe-90b5-d74c095c812e\") " pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.337088 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.337004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:39.502581 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:39.502551 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd"] Jul 15 19:05:39.504641 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:05:39.504610 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736ad68d_b406_44fe_90b5_d74c095c812e.slice/crio-a63d6d3fe2747b71773ef0648a5b57652b0af37eccadcbc5fefeb7e3b21f78aa WatchSource:0}: Error finding container a63d6d3fe2747b71773ef0648a5b57652b0af37eccadcbc5fefeb7e3b21f78aa: Status 404 returned error can't find the container with id a63d6d3fe2747b71773ef0648a5b57652b0af37eccadcbc5fefeb7e3b21f78aa Jul 15 19:05:40.465600 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:40.465560 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" event={"ID":"736ad68d-b406-44fe-90b5-d74c095c812e","Type":"ContainerStarted","Data":"a63d6d3fe2747b71773ef0648a5b57652b0af37eccadcbc5fefeb7e3b21f78aa"} Jul 15 19:05:43.481424 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:43.481388 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" event={"ID":"736ad68d-b406-44fe-90b5-d74c095c812e","Type":"ContainerStarted","Data":"712b3cf1176aa55ac0a9a3c0dfe8c250e14a604189bb4d9f0fefa249ed193655"} Jul 15 19:05:44.486941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:44.486897 2572 generic.go:356] "Generic (PLEG): container finished" podID="736ad68d-b406-44fe-90b5-d74c095c812e" containerID="712b3cf1176aa55ac0a9a3c0dfe8c250e14a604189bb4d9f0fefa249ed193655" exitCode=0 Jul 15 19:05:44.487355 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:44.486979 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" event={"ID":"736ad68d-b406-44fe-90b5-d74c095c812e","Type":"ContainerDied","Data":"712b3cf1176aa55ac0a9a3c0dfe8c250e14a604189bb4d9f0fefa249ed193655"} Jul 15 19:05:45.493449 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:45.493412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" event={"ID":"736ad68d-b406-44fe-90b5-d74c095c812e","Type":"ContainerStarted","Data":"11e553d743776a0a336ea57d3d7baffa4ab9df640bdfe13e3d07602aff64bd2e"} Jul 15 19:05:45.493823 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:45.493524 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:05:45.517170 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:45.517118 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" podStartSLOduration=2.8267708320000002 podStartE2EDuration="7.517104911s" totalImagesPullingTime="3.592828119s" totalInitContainerRuntime="1.09750596s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:05:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:05:45.515694528 +0000 UTC m=+314.982668854" watchObservedRunningTime="2026-07-15 19:05:45.517104911 +0000 UTC m=+314.984079221" Jul 15 19:05:48.457701 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:48.457671 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8659b4c98-fdkvl" Jul 15 19:05:56.499797 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:05:56.499764 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55c89bff45-p4cdd" Jul 15 19:06:06.252505 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.252471 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn"] Jul 15 19:06:06.257110 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.257091 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.261033 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.261009 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 19:06:06.261210 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.261193 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 19:06:06.261347 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.261284 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sgqpr\"" Jul 15 19:06:06.261550 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.261535 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 19:06:06.261741 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.261724 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 19:06:06.285803 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.285781 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn"] Jul 15 19:06:06.358154 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4z64\" (UniqueName: \"kubernetes.io/projected/bfd3bd0a-8538-411c-b895-a53e947b90e4-kube-api-access-s4z64\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.358335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358163 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.358335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358196 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.358335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358220 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd3bd0a-8538-411c-b895-a53e947b90e4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.358335 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358259 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.358481 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358335 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.358481 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.358419 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459087 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459323 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459142 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4z64\" (UniqueName: \"kubernetes.io/projected/bfd3bd0a-8538-411c-b895-a53e947b90e4-kube-api-access-s4z64\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459323 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459323 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459194 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459323 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd3bd0a-8538-411c-b895-a53e947b90e4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459323 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459254 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459323 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.459985 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.459961 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.462164 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.462136 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd3bd0a-8538-411c-b895-a53e947b90e4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.462478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.462457 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.462529 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.462464 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.462564 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.462541 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.468444 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.468421 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd3bd0a-8538-411c-b895-a53e947b90e4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.472846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.472828 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4z64\" (UniqueName: \"kubernetes.io/projected/bfd3bd0a-8538-411c-b895-a53e947b90e4-kube-api-access-s4z64\") pod \"istiod-openshift-gateway-94bb8fbfd-8c5qn\" (UID: \"bfd3bd0a-8538-411c-b895-a53e947b90e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.566375 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.566306 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:06.697580 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:06.697555 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn"] Jul 15 19:06:06.699978 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:06:06.699951 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd3bd0a_8538_411c_b895_a53e947b90e4.slice/crio-3e03c234ba5711c25fa8e51ce36d4c541b4321841a4c17083c0c83fbb54b09ae WatchSource:0}: Error finding container 3e03c234ba5711c25fa8e51ce36d4c541b4321841a4c17083c0c83fbb54b09ae: Status 404 returned error can't find the container with id 3e03c234ba5711c25fa8e51ce36d4c541b4321841a4c17083c0c83fbb54b09ae Jul 15 19:06:07.595973 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:07.595926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" event={"ID":"bfd3bd0a-8538-411c-b895-a53e947b90e4","Type":"ContainerStarted","Data":"3e03c234ba5711c25fa8e51ce36d4c541b4321841a4c17083c0c83fbb54b09ae"} Jul 15 19:06:09.267692 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:09.267656 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 19:06:09.267928 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:09.267728 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 19:06:09.608347 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:09.608307 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" event={"ID":"bfd3bd0a-8538-411c-b895-a53e947b90e4","Type":"ContainerStarted","Data":"493c5c406f0990d9d9348a2e3836f1afd74277a96ff77c10b1d064b4b64819d7"} Jul 15 19:06:09.608527 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:09.608380 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:06:09.675158 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:09.675109 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" podStartSLOduration=1.109525542 podStartE2EDuration="3.675092427s" totalImagesPullingTime="2.565566885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:06:09.67039459 +0000 UTC m=+339.137368900" watchObservedRunningTime="2026-07-15 19:06:09.675092427 +0000 UTC m=+339.142066736" Jul 15 19:06:10.614158 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:06:10.614132 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8c5qn" Jul 15 19:07:37.898003 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:07:37.897969 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c476cc6d5-fmv6r"] Jul 15 19:08:02.918071 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:02.917974 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c476cc6d5-fmv6r" podUID="dc168814-5f4e-4f59-8c63-ab8c184b8ef7" containerName="console" containerID="cri-o://9c0683e300690c56b28ac00da8ba406fb33500f74b168c6cf66909bc5a1fdd02" gracePeriod=15 Jul 15 19:08:03.118059 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.118028 2572 generic.go:356] "Generic (PLEG): container finished" podID="dc168814-5f4e-4f59-8c63-ab8c184b8ef7" containerID="9c0683e300690c56b28ac00da8ba406fb33500f74b168c6cf66909bc5a1fdd02" exitCode=0 Jul 15 19:08:03.118215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.118104 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c476cc6d5-fmv6r" event={"ID":"dc168814-5f4e-4f59-8c63-ab8c184b8ef7","Type":"ContainerDied","Data":"9c0683e300690c56b28ac00da8ba406fb33500f74b168c6cf66909bc5a1fdd02"} Jul 15 19:08:03.169994 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.169932 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:08:03.228799 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.228770 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-trusted-ca-bundle\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.228982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.228808 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-service-ca\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.228982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.228840 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-config\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.228982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.228862 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5jhg\" (UniqueName: \"kubernetes.io/projected/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-kube-api-access-d5jhg\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.228982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.228887 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-serving-cert\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.228982 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.228906 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-oauth-serving-cert\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.229301 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.229274 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:08:03.229449 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.229430 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:08:03.229534 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.229427 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-config" (OuterVolumeSpecName: "console-config") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:08:03.229534 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.229476 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:08:03.231141 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.231116 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:08:03.231141 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.231127 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-kube-api-access-d5jhg" (OuterVolumeSpecName: "kube-api-access-d5jhg") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "kube-api-access-d5jhg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:08:03.329377 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329338 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-oauth-config\") pod \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\" (UID: \"dc168814-5f4e-4f59-8c63-ab8c184b8ef7\") " Jul 15 19:08:03.329573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329517 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-trusted-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:03.329573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329527 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:03.329573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329538 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:03.329573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329546 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5jhg\" (UniqueName: \"kubernetes.io/projected/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-kube-api-access-d5jhg\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:03.329573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329555 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-serving-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:03.329573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.329563 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-oauth-serving-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:03.331434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.331408 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc168814-5f4e-4f59-8c63-ab8c184b8ef7" (UID: "dc168814-5f4e-4f59-8c63-ab8c184b8ef7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:08:03.430176 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:03.430095 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc168814-5f4e-4f59-8c63-ab8c184b8ef7-console-oauth-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:04.124038 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.124002 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c476cc6d5-fmv6r" Jul 15 19:08:04.124528 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.124007 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c476cc6d5-fmv6r" event={"ID":"dc168814-5f4e-4f59-8c63-ab8c184b8ef7","Type":"ContainerDied","Data":"87226c9d2f127625cd8d9da54f7ccb6861a2bfa918e68b6dcaf4c58b800d7185"} Jul 15 19:08:04.124528 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.124141 2572 scope.go:117] "RemoveContainer" containerID="9c0683e300690c56b28ac00da8ba406fb33500f74b168c6cf66909bc5a1fdd02" Jul 15 19:08:04.147521 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.147494 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c476cc6d5-fmv6r"] Jul 15 19:08:04.159423 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.158893 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c476cc6d5-fmv6r"] Jul 15 19:08:04.723622 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.723586 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m"] Jul 15 19:08:04.723961 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.723946 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc168814-5f4e-4f59-8c63-ab8c184b8ef7" containerName="console" Jul 15 19:08:04.724008 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.723963 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc168814-5f4e-4f59-8c63-ab8c184b8ef7" containerName="console" Jul 15 19:08:04.724042 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.724027 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc168814-5f4e-4f59-8c63-ab8c184b8ef7" containerName="console" Jul 15 19:08:04.728214 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.728189 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.731119 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.731098 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 19:08:04.731817 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.731802 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 19:08:04.731937 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.731812 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-klgdv\"" Jul 15 19:08:04.740972 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.740950 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbswm\" (UniqueName: \"kubernetes.io/projected/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-kube-api-access-tbswm\") pod \"kuadrant-operator-controller-manager-86878c769f-fw94m\" (UID: \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.741075 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.740980 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fw94m\" (UID: \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.783967 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.783933 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m"] Jul 15 19:08:04.794707 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.794683 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m"] Jul 15 19:08:04.794938 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:04.794920 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-tbswm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" Jul 15 19:08:04.806367 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.806342 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m"] Jul 15 19:08:04.812592 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:04.812541 2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-15T19:08:04Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-15T19:08:04Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-15T19:08:04Z\\\",\\\"message\\\":\\\"containers with unready status: [manager]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-15T19:08:04Z\\\",\\\"message\\\":\\\"containers with unready status: [manager]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/kuadrant/kuadrant-operator:v1.4.2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/tmp/kuadrant\\\",\\\"name\\\":\\\"extensions-socket-volume\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tbswm\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.137.168\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.137.168\\\"}],\\\"observedGeneration\\\":1,\\\"startTime\\\":\\\"2026-07-15T19:08:04Z\\\"}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-86878c769f-fw94m\": pods \"kuadrant-operator-controller-manager-86878c769f-fw94m\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.841962 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.841929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbswm\" (UniqueName: \"kubernetes.io/projected/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-kube-api-access-tbswm\") pod \"kuadrant-operator-controller-manager-86878c769f-fw94m\" (UID: \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.842108 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.841968 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fw94m\" (UID: \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.842411 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.842391 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fw94m\" (UID: \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.852568 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:04.852541 2572 projected.go:196] Error preparing data for projected volume kube-api-access-tbswm for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-137-168.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object Jul 15 19:08:04.852709 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:04.852622 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-kube-api-access-tbswm podName:e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54 nodeName:}" failed. No retries permitted until 2026-07-15 19:08:05.352605943 +0000 UTC m=+454.819580237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tbswm" (UniqueName: "kubernetes.io/projected/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-kube-api-access-tbswm") pod "kuadrant-operator-controller-manager-86878c769f-fw94m" (UID: "e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-137-168.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object Jul 15 19:08:04.862478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.862449 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc"] Jul 15 19:08:04.865796 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.865778 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:04.868999 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:04.868973 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-fw94m\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:04.896608 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.896582 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc"] Jul 15 19:08:04.943193 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.943152 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70835e8-8140-401b-b296-8d8305334c36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rz5qc\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:04.943363 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:04.943260 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8td5p\" (UniqueName: \"kubernetes.io/projected/e70835e8-8140-401b-b296-8d8305334c36-kube-api-access-8td5p\") pod \"kuadrant-operator-controller-manager-86878c769f-rz5qc\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:05.033480 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.033405 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc168814-5f4e-4f59-8c63-ab8c184b8ef7" path="/var/lib/kubelet/pods/dc168814-5f4e-4f59-8c63-ab8c184b8ef7/volumes" Jul 15 19:08:05.043846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.043822 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70835e8-8140-401b-b296-8d8305334c36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rz5qc\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:05.043970 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.043863 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8td5p\" (UniqueName: \"kubernetes.io/projected/e70835e8-8140-401b-b296-8d8305334c36-kube-api-access-8td5p\") pod \"kuadrant-operator-controller-manager-86878c769f-rz5qc\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:05.044215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.044193 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70835e8-8140-401b-b296-8d8305334c36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rz5qc\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:05.067791 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.067764 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8td5p\" (UniqueName: \"kubernetes.io/projected/e70835e8-8140-401b-b296-8d8305334c36-kube-api-access-8td5p\") pod \"kuadrant-operator-controller-manager-86878c769f-rz5qc\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:05.128645 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.128618 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:05.131533 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:05.131507 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-fw94m\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:05.135468 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.135450 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:05.140654 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:05.140630 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-fw94m\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:05.144930 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.144917 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-extensions-socket-volume\") pod \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\" (UID: \"e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54\") " Jul 15 19:08:05.145049 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.145032 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbswm\" (UniqueName: \"kubernetes.io/projected/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-kube-api-access-tbswm\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:05.145156 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.145137 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" (UID: "e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:08:05.175406 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.175385 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:05.246024 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.245990 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54-extensions-socket-volume\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:08:05.320624 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:05.320599 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc"] Jul 15 19:08:05.322946 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:08:05.322918 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70835e8_8140_401b_b296_8d8305334c36.slice/crio-7ad0a06132c78698e53d7d4c581a6502edac9c2456795ed84fa5bf0b36e9fff4 WatchSource:0}: Error finding container 7ad0a06132c78698e53d7d4c581a6502edac9c2456795ed84fa5bf0b36e9fff4: Status 404 returned error can't find the container with id 7ad0a06132c78698e53d7d4c581a6502edac9c2456795ed84fa5bf0b36e9fff4 Jul 15 19:08:06.133947 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:06.133910 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" event={"ID":"e70835e8-8140-401b-b296-8d8305334c36","Type":"ContainerStarted","Data":"7ad0a06132c78698e53d7d4c581a6502edac9c2456795ed84fa5bf0b36e9fff4"} Jul 15 19:08:06.134324 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:06.133975 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:06.138648 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:06.138615 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-fw94m\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:06.144502 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:08:06.144473 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-fw94m\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fw94m" Jul 15 19:08:07.034080 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:07.034048 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54" path="/var/lib/kubelet/pods/e6e1ce79-16c1-4fec-a8ed-9cc3ad80de54/volumes" Jul 15 19:08:12.164544 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:12.164508 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" event={"ID":"e70835e8-8140-401b-b296-8d8305334c36","Type":"ContainerStarted","Data":"450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0"} Jul 15 19:08:12.164933 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:12.164653 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:08:12.191268 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:12.191152 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" podStartSLOduration=1.5879501120000001 podStartE2EDuration="8.191132931s" totalImagesPullingTime="6.603182819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:08:12.189413055 +0000 UTC m=+461.656387367" watchObservedRunningTime="2026-07-15 19:08:12.191132931 +0000 UTC m=+461.658107242" Jul 15 19:08:23.171673 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:08:23.171640 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:10:00.102963 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.102928 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-thpfd"] Jul 15 19:10:00.105394 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.105376 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.108291 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.108271 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lsnfb\"" Jul 15 19:10:00.108406 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.108270 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 19:10:00.117002 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.116980 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-thpfd"] Jul 15 19:10:00.201873 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.201833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-thpfd\" (UID: \"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.202078 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.201897 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd5ms\" (UniqueName: \"kubernetes.io/projected/8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48-kube-api-access-jd5ms\") pod \"payload-processing-f9b8d9b9b-thpfd\" (UID: \"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.302939 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.302900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd5ms\" (UniqueName: \"kubernetes.io/projected/8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48-kube-api-access-jd5ms\") pod \"payload-processing-f9b8d9b9b-thpfd\" (UID: \"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.303106 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.302978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-thpfd\" (UID: \"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.303663 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.303643 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-thpfd\" (UID: \"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.315432 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.315406 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd5ms\" (UniqueName: \"kubernetes.io/projected/8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48-kube-api-access-jd5ms\") pod \"payload-processing-f9b8d9b9b-thpfd\" (UID: \"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.416225 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.416131 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:00.551668 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.551631 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-thpfd"] Jul 15 19:10:00.554154 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:10:00.554124 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8e8a5d_50ae_4148_aca3_afbc6a4b5d48.slice/crio-e41073c8faa3460da9f54dec568e0febfda2e31c6d037a58401081e5f2b1cc44 WatchSource:0}: Error finding container e41073c8faa3460da9f54dec568e0febfda2e31c6d037a58401081e5f2b1cc44: Status 404 returned error can't find the container with id e41073c8faa3460da9f54dec568e0febfda2e31c6d037a58401081e5f2b1cc44 Jul 15 19:10:00.668924 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:00.668828 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" event={"ID":"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48","Type":"ContainerStarted","Data":"e41073c8faa3460da9f54dec568e0febfda2e31c6d037a58401081e5f2b1cc44"} Jul 15 19:10:02.679049 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:02.679004 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" event={"ID":"8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48","Type":"ContainerStarted","Data":"591a1f58c4d6cbbb57be0248e4d18e52854fd70d8f9214e916057dc0c3f363a2"} Jul 15 19:10:02.679456 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:02.679075 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:02.695895 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:02.695838 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" podStartSLOduration=1.200593038 podStartE2EDuration="2.695820959s" totalImagesPullingTime="1.495227921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:10:02.695092756 +0000 UTC m=+572.162067069" watchObservedRunningTime="2026-07-15 19:10:02.695820959 +0000 UTC m=+572.162795311" Jul 15 19:10:13.685425 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:13.685391 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-thpfd" Jul 15 19:10:24.223742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.223710 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr"] Jul 15 19:10:24.226268 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.226229 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.229059 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.229032 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 19:10:24.229199 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.229157 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 19:10:24.230175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.230154 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6xgvb\"" Jul 15 19:10:24.230263 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.230190 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 19:10:24.243661 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.243639 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr"] Jul 15 19:10:24.286944 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.286914 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lnvm\" (UniqueName: \"kubernetes.io/projected/0c8980fe-caa3-4142-8d74-2a3db5d73656-kube-api-access-9lnvm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.287089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.286967 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.287089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.286991 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.287089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.287026 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.287089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.287066 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.287089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.287086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c8980fe-caa3-4142-8d74-2a3db5d73656-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387501 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387464 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387501 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387505 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387526 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387591 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c8980fe-caa3-4142-8d74-2a3db5d73656-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387635 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnvm\" (UniqueName: \"kubernetes.io/projected/0c8980fe-caa3-4142-8d74-2a3db5d73656-kube-api-access-9lnvm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.387940 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.387908 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.388042 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.388008 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.388042 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.388017 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.389777 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.389751 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c8980fe-caa3-4142-8d74-2a3db5d73656-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.390067 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.390047 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c8980fe-caa3-4142-8d74-2a3db5d73656-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.396486 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.396461 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnvm\" (UniqueName: \"kubernetes.io/projected/0c8980fe-caa3-4142-8d74-2a3db5d73656-kube-api-access-9lnvm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gffgr\" (UID: \"0c8980fe-caa3-4142-8d74-2a3db5d73656\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.536584 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.536492 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:24.668559 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.668532 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr"] Jul 15 19:10:24.670826 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:10:24.670791 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8980fe_caa3_4142_8d74_2a3db5d73656.slice/crio-0f442d84f7fbdb3882ae6bcd77a5df47cfa92e59db256b8c27578f1403aba9ec WatchSource:0}: Error finding container 0f442d84f7fbdb3882ae6bcd77a5df47cfa92e59db256b8c27578f1403aba9ec: Status 404 returned error can't find the container with id 0f442d84f7fbdb3882ae6bcd77a5df47cfa92e59db256b8c27578f1403aba9ec Jul 15 19:10:24.778307 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:24.778266 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" event={"ID":"0c8980fe-caa3-4142-8d74-2a3db5d73656","Type":"ContainerStarted","Data":"0f442d84f7fbdb3882ae6bcd77a5df47cfa92e59db256b8c27578f1403aba9ec"} Jul 15 19:10:26.789000 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:26.788964 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" event={"ID":"0c8980fe-caa3-4142-8d74-2a3db5d73656","Type":"ContainerStarted","Data":"212a6279c5a36844bacfde704d0ee33a4d9b797186989c0961b9d3541dfdbb8a"} Jul 15 19:10:26.807583 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:26.807516 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" podStartSLOduration=1.203596022 podStartE2EDuration="2.807495857s" totalImagesPullingTime="1.603899835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:10:26.805481672 +0000 UTC m=+596.272455983" watchObservedRunningTime="2026-07-15 19:10:26.807495857 +0000 UTC m=+596.274470166" Jul 15 19:10:31.002352 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:31.002326 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:10:31.002946 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:31.002877 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:10:31.009160 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:31.009129 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:10:31.009846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:31.009825 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:10:34.537535 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:34.537499 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:34.537928 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:34.537560 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:34.549435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:34.549409 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:10:34.837401 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:10:34.837316 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gffgr" Jul 15 19:15:00.167717 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.167675 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z"] Jul 15 19:15:00.171318 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.171293 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" Jul 15 19:15:00.175346 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.175316 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jwf9b\"" Jul 15 19:15:00.181528 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.181505 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" Jul 15 19:15:00.182800 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.182776 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z"] Jul 15 19:15:00.327086 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.327048 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z"] Jul 15 19:15:00.331007 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:15:00.330970 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16bf84bb_016a_4c36_804d_b775f456154a.slice/crio-69ec7f933d344a2535200c339e738ebc4104c4fd6ec4eaeb1623e80f73220318 WatchSource:0}: Error finding container 69ec7f933d344a2535200c339e738ebc4104c4fd6ec4eaeb1623e80f73220318: Status 404 returned error can't find the container with id 69ec7f933d344a2535200c339e738ebc4104c4fd6ec4eaeb1623e80f73220318 Jul 15 19:15:00.333048 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:00.333026 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:15:01.052249 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:01.052180 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" event={"ID":"16bf84bb-016a-4c36-804d-b775f456154a","Type":"ContainerStarted","Data":"69ec7f933d344a2535200c339e738ebc4104c4fd6ec4eaeb1623e80f73220318"} Jul 15 19:15:04.067642 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:04.067555 2572 generic.go:356] "Generic (PLEG): container finished" podID="16bf84bb-016a-4c36-804d-b775f456154a" containerID="0eb3381939d67719bdd689f6403e4abd76efa1a4526b4523ea3097def090b943" exitCode=0 Jul 15 19:15:04.067985 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:04.067640 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" event={"ID":"16bf84bb-016a-4c36-804d-b775f456154a","Type":"ContainerDied","Data":"0eb3381939d67719bdd689f6403e4abd76efa1a4526b4523ea3097def090b943"} Jul 15 19:15:05.207407 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:05.207379 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" Jul 15 19:15:06.077799 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:06.077766 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" Jul 15 19:15:06.077976 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:06.077801 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735715-6v94z" event={"ID":"16bf84bb-016a-4c36-804d-b775f456154a","Type":"ContainerDied","Data":"69ec7f933d344a2535200c339e738ebc4104c4fd6ec4eaeb1623e80f73220318"} Jul 15 19:15:06.077976 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:06.077857 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69ec7f933d344a2535200c339e738ebc4104c4fd6ec4eaeb1623e80f73220318" Jul 15 19:15:31.031976 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:31.031892 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:15:31.033797 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:31.033769 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:15:31.039972 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:31.039950 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:15:31.041108 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:15:31.041087 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:18:13.212212 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:13.212179 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc"] Jul 15 19:18:13.212708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:13.212402 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" podUID="e70835e8-8140-401b-b296-8d8305334c36" containerName="manager" containerID="cri-o://450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0" gracePeriod=10 Jul 15 19:18:15.464770 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.464739 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:18:15.523810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.523733 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8td5p\" (UniqueName: \"kubernetes.io/projected/e70835e8-8140-401b-b296-8d8305334c36-kube-api-access-8td5p\") pod \"e70835e8-8140-401b-b296-8d8305334c36\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " Jul 15 19:18:15.523810 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.523806 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70835e8-8140-401b-b296-8d8305334c36-extensions-socket-volume\") pod \"e70835e8-8140-401b-b296-8d8305334c36\" (UID: \"e70835e8-8140-401b-b296-8d8305334c36\") " Jul 15 19:18:15.524184 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.524155 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70835e8-8140-401b-b296-8d8305334c36-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e70835e8-8140-401b-b296-8d8305334c36" (UID: "e70835e8-8140-401b-b296-8d8305334c36"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:18:15.526077 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.526046 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70835e8-8140-401b-b296-8d8305334c36-kube-api-access-8td5p" (OuterVolumeSpecName: "kube-api-access-8td5p") pod "e70835e8-8140-401b-b296-8d8305334c36" (UID: "e70835e8-8140-401b-b296-8d8305334c36"). InnerVolumeSpecName "kube-api-access-8td5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:18:15.625140 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.625103 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8td5p\" (UniqueName: \"kubernetes.io/projected/e70835e8-8140-401b-b296-8d8305334c36-kube-api-access-8td5p\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:18:15.625140 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.625133 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70835e8-8140-401b-b296-8d8305334c36-extensions-socket-volume\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:18:15.966162 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.966123 2572 generic.go:356] "Generic (PLEG): container finished" podID="e70835e8-8140-401b-b296-8d8305334c36" containerID="450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0" exitCode=0 Jul 15 19:18:15.966377 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.966169 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" Jul 15 19:18:15.966377 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.966194 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" event={"ID":"e70835e8-8140-401b-b296-8d8305334c36","Type":"ContainerDied","Data":"450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0"} Jul 15 19:18:15.966377 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.966272 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc" event={"ID":"e70835e8-8140-401b-b296-8d8305334c36","Type":"ContainerDied","Data":"7ad0a06132c78698e53d7d4c581a6502edac9c2456795ed84fa5bf0b36e9fff4"} Jul 15 19:18:15.966377 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.966304 2572 scope.go:117] "RemoveContainer" containerID="450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0" Jul 15 19:18:15.977391 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.977364 2572 scope.go:117] "RemoveContainer" containerID="450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0" Jul 15 19:18:15.977786 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:18:15.977763 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0\": container with ID starting with 450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0 not found: ID does not exist" containerID="450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0" Jul 15 19:18:15.977846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.977795 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0"} err="failed to get container status \"450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0\": rpc error: code = NotFound desc = could not find container \"450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0\": container with ID starting with 450b0a22ef53bd0e1c57ff0b0b15d3164af1307e37f1275af1f440e8abfe9ad0 not found: ID does not exist" Jul 15 19:18:15.991867 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.991838 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc"] Jul 15 19:18:15.997434 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:15.997413 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rz5qc"] Jul 15 19:18:17.032838 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:18:17.032803 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70835e8-8140-401b-b296-8d8305334c36" path="/var/lib/kubelet/pods/e70835e8-8140-401b-b296-8d8305334c36/volumes" Jul 15 19:19:14.606040 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606003 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk"] Jul 15 19:19:14.606532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606352 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16bf84bb-016a-4c36-804d-b775f456154a" containerName="cleanup" Jul 15 19:19:14.606532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606365 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="16bf84bb-016a-4c36-804d-b775f456154a" containerName="cleanup" Jul 15 19:19:14.606532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606378 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70835e8-8140-401b-b296-8d8305334c36" containerName="manager" Jul 15 19:19:14.606532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606383 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70835e8-8140-401b-b296-8d8305334c36" containerName="manager" Jul 15 19:19:14.606532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606445 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="16bf84bb-016a-4c36-804d-b775f456154a" containerName="cleanup" Jul 15 19:19:14.606532 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.606455 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70835e8-8140-401b-b296-8d8305334c36" containerName="manager" Jul 15 19:19:14.609398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.609380 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.635275 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.635245 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9c7475-0712-4567-8833-2168614b3547-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7whk\" (UID: \"fa9c7475-0712-4567-8833-2168614b3547\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.635405 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.635338 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxsp\" (UniqueName: \"kubernetes.io/projected/fa9c7475-0712-4567-8833-2168614b3547-kube-api-access-7nxsp\") pod \"kuadrant-operator-controller-manager-86878c769f-c7whk\" (UID: \"fa9c7475-0712-4567-8833-2168614b3547\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.635992 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.635973 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 19:19:14.636921 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.636905 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 19:19:14.637008 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.636994 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-klgdv\"" Jul 15 19:19:14.669089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.668365 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk"] Jul 15 19:19:14.736112 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.736077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxsp\" (UniqueName: \"kubernetes.io/projected/fa9c7475-0712-4567-8833-2168614b3547-kube-api-access-7nxsp\") pod \"kuadrant-operator-controller-manager-86878c769f-c7whk\" (UID: \"fa9c7475-0712-4567-8833-2168614b3547\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.736291 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.736146 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9c7475-0712-4567-8833-2168614b3547-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7whk\" (UID: \"fa9c7475-0712-4567-8833-2168614b3547\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.736573 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.736553 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa9c7475-0712-4567-8833-2168614b3547-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7whk\" (UID: \"fa9c7475-0712-4567-8833-2168614b3547\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.763067 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.763034 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxsp\" (UniqueName: \"kubernetes.io/projected/fa9c7475-0712-4567-8833-2168614b3547-kube-api-access-7nxsp\") pod \"kuadrant-operator-controller-manager-86878c769f-c7whk\" (UID: \"fa9c7475-0712-4567-8833-2168614b3547\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:14.919069 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:14.918981 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:15.060000 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:15.059976 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk"] Jul 15 19:19:15.065814 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:19:15.065779 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9c7475_0712_4567_8833_2168614b3547.slice/crio-2ccb7cb3d520a8fbd2df2efc37c99473e080e0313ef71701abba537690cb9e4b WatchSource:0}: Error finding container 2ccb7cb3d520a8fbd2df2efc37c99473e080e0313ef71701abba537690cb9e4b: Status 404 returned error can't find the container with id 2ccb7cb3d520a8fbd2df2efc37c99473e080e0313ef71701abba537690cb9e4b Jul 15 19:19:15.238093 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:15.238010 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" event={"ID":"fa9c7475-0712-4567-8833-2168614b3547","Type":"ContainerStarted","Data":"10892676872cf91f5c51aade29bfbfcf25b9b1c354d941d8ee06d35511fa3ccc"} Jul 15 19:19:15.238093 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:15.238061 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" event={"ID":"fa9c7475-0712-4567-8833-2168614b3547","Type":"ContainerStarted","Data":"2ccb7cb3d520a8fbd2df2efc37c99473e080e0313ef71701abba537690cb9e4b"} Jul 15 19:19:15.238319 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:15.238137 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:19:15.271639 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:15.271590 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" podStartSLOduration=1.271573753 podStartE2EDuration="1.271573753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:19:15.269797686 +0000 UTC m=+1124.736771998" watchObservedRunningTime="2026-07-15 19:19:15.271573753 +0000 UTC m=+1124.738548064" Jul 15 19:19:26.244744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:19:26.244708 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7whk" Jul 15 19:20:31.060453 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:20:31.060426 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:20:31.063245 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:20:31.063208 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:20:31.068722 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:20:31.068686 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:20:31.071386 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:20:31.071363 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:25:31.089252 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:25:31.089209 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:25:31.093378 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:25:31.093354 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:25:31.096732 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:25:31.096707 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:25:31.100700 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:25:31.100679 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:26:24.993872 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:24.993785 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv"] Jul 15 19:26:24.997722 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:24.997702 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.001295 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.001268 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bdsfz\"" Jul 15 19:26:25.001463 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.001315 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:26:25.001463 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.001267 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-38656213-serving-cert\"" Jul 15 19:26:25.002908 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.002891 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 19:26:25.021424 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.021403 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.021533 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.021434 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-openshift-service-ca\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.021533 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.021456 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d9x4\" (UniqueName: \"kubernetes.io/projected/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-kube-api-access-5d9x4\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.034849 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.034822 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv"] Jul 15 19:26:25.123299 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.123261 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.123540 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.123520 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-openshift-service-ca\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.123666 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.123651 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d9x4\" (UniqueName: \"kubernetes.io/projected/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-kube-api-access-5d9x4\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.124223 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:26:25.124202 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-38656213-serving-cert: secret "maas-api-e2e-derive-38656213-serving-cert" not found Jul 15 19:26:25.124525 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:26:25.124511 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls podName:ededf37f-7e9d-41c8-b2bb-9425834aa8fd nodeName:}" failed. No retries permitted until 2026-07-15 19:26:25.624490934 +0000 UTC m=+1555.091465237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls") pod "maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" (UID: "ededf37f-7e9d-41c8-b2bb-9425834aa8fd") : secret "maas-api-e2e-derive-38656213-serving-cert" not found Jul 15 19:26:25.125406 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.125384 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-openshift-service-ca\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.155279 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.155223 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d9x4\" (UniqueName: \"kubernetes.io/projected/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-kube-api-access-5d9x4\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.628325 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.628287 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.630789 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.630759 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls\") pod \"maas-api-e2e-derive-38656213-5d9f745c48-jg4bv\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:25.908767 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:25.908670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:26.069350 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:26.069324 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv"] Jul 15 19:26:26.071816 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:26:26.071785 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podededf37f_7e9d_41c8_b2bb_9425834aa8fd.slice/crio-5fcbc27a4310f5d7b8aeb112a58533980c944445398b2b5dc04f9d7326c7693c WatchSource:0}: Error finding container 5fcbc27a4310f5d7b8aeb112a58533980c944445398b2b5dc04f9d7326c7693c: Status 404 returned error can't find the container with id 5fcbc27a4310f5d7b8aeb112a58533980c944445398b2b5dc04f9d7326c7693c Jul 15 19:26:26.073218 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:26.073194 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:26:26.254972 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:26.254890 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" event={"ID":"ededf37f-7e9d-41c8-b2bb-9425834aa8fd","Type":"ContainerStarted","Data":"5fcbc27a4310f5d7b8aeb112a58533980c944445398b2b5dc04f9d7326c7693c"} Jul 15 19:26:29.271313 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:29.271268 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" event={"ID":"ededf37f-7e9d-41c8-b2bb-9425834aa8fd","Type":"ContainerStarted","Data":"f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563"} Jul 15 19:26:29.271771 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:29.271360 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:29.289432 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:29.289382 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" podStartSLOduration=3.043753085 podStartE2EDuration="5.28936765s" totalImagesPullingTime="2.245614565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:29.288333226 +0000 UTC m=+1558.755307538" watchObservedRunningTime="2026-07-15 19:26:29.28936765 +0000 UTC m=+1558.756341960" Jul 15 19:26:35.283014 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:35.282983 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:36.268793 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.268759 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv"] Jul 15 19:26:36.269577 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.269539 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" podUID="ededf37f-7e9d-41c8-b2bb-9425834aa8fd" containerName="maas-api" containerID="cri-o://f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563" gracePeriod=30 Jul 15 19:26:36.518248 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.518211 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:36.628905 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.628875 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d9x4\" (UniqueName: \"kubernetes.io/projected/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-kube-api-access-5d9x4\") pod \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " Jul 15 19:26:36.629069 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.628910 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls\") pod \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " Jul 15 19:26:36.629069 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.628940 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-openshift-service-ca\") pod \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\" (UID: \"ededf37f-7e9d-41c8-b2bb-9425834aa8fd\") " Jul 15 19:26:36.629389 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.629356 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ededf37f-7e9d-41c8-b2bb-9425834aa8fd" (UID: "ededf37f-7e9d-41c8-b2bb-9425834aa8fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:26:36.631128 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.631097 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ededf37f-7e9d-41c8-b2bb-9425834aa8fd" (UID: "ededf37f-7e9d-41c8-b2bb-9425834aa8fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:36.631214 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.631140 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-kube-api-access-5d9x4" (OuterVolumeSpecName: "kube-api-access-5d9x4") pod "ededf37f-7e9d-41c8-b2bb-9425834aa8fd" (UID: "ededf37f-7e9d-41c8-b2bb-9425834aa8fd"). InnerVolumeSpecName "kube-api-access-5d9x4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:36.730015 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.729979 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d9x4\" (UniqueName: \"kubernetes.io/projected/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-kube-api-access-5d9x4\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:26:36.730015 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.730014 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:26:36.730192 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:36.730027 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ededf37f-7e9d-41c8-b2bb-9425834aa8fd-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:26:37.311242 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.311140 2572 generic.go:356] "Generic (PLEG): container finished" podID="ededf37f-7e9d-41c8-b2bb-9425834aa8fd" containerID="f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563" exitCode=0 Jul 15 19:26:37.311242 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.311213 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" Jul 15 19:26:37.311418 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.311222 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" event={"ID":"ededf37f-7e9d-41c8-b2bb-9425834aa8fd","Type":"ContainerDied","Data":"f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563"} Jul 15 19:26:37.311418 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.311300 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv" event={"ID":"ededf37f-7e9d-41c8-b2bb-9425834aa8fd","Type":"ContainerDied","Data":"5fcbc27a4310f5d7b8aeb112a58533980c944445398b2b5dc04f9d7326c7693c"} Jul 15 19:26:37.311418 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.311323 2572 scope.go:117] "RemoveContainer" containerID="f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563" Jul 15 19:26:37.319886 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.319867 2572 scope.go:117] "RemoveContainer" containerID="f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563" Jul 15 19:26:37.320187 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:26:37.320167 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563\": container with ID starting with f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563 not found: ID does not exist" containerID="f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563" Jul 15 19:26:37.320280 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.320197 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563"} err="failed to get container status \"f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563\": rpc error: code = NotFound desc = could not find container \"f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563\": container with ID starting with f38217a22f4ad42627951e2b0b64b898034fe8be5c828d182a39d2f641af1563 not found: ID does not exist" Jul 15 19:26:37.336793 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.336770 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv"] Jul 15 19:26:37.340164 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:37.340143 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-38656213-5d9f745c48-jg4bv"] Jul 15 19:26:39.033172 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:26:39.033133 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ededf37f-7e9d-41c8-b2bb-9425834aa8fd" path="/var/lib/kubelet/pods/ededf37f-7e9d-41c8-b2bb-9425834aa8fd/volumes" Jul 15 19:28:22.660290 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.660253 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8"] Jul 15 19:28:22.660772 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.660755 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ededf37f-7e9d-41c8-b2bb-9425834aa8fd" containerName="maas-api" Jul 15 19:28:22.660818 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.660776 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ededf37f-7e9d-41c8-b2bb-9425834aa8fd" containerName="maas-api" Jul 15 19:28:22.660864 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.660854 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ededf37f-7e9d-41c8-b2bb-9425834aa8fd" containerName="maas-api" Jul 15 19:28:22.664127 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.664109 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.666928 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.666889 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e84500d1-serving-cert\"" Jul 15 19:28:22.667047 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.666926 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:28:22.668081 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.668066 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bdsfz\"" Jul 15 19:28:22.668154 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.668103 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 19:28:22.679647 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.679600 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8"] Jul 15 19:28:22.717470 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.717439 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.717647 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.717513 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b339b9bf-92af-4c49-b585-d74ad18147a8-openshift-service-ca\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.717647 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.717535 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hzh9\" (UniqueName: \"kubernetes.io/projected/b339b9bf-92af-4c49-b585-d74ad18147a8-kube-api-access-4hzh9\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.818016 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.817982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b339b9bf-92af-4c49-b585-d74ad18147a8-openshift-service-ca\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.818016 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.818019 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hzh9\" (UniqueName: \"kubernetes.io/projected/b339b9bf-92af-4c49-b585-d74ad18147a8-kube-api-access-4hzh9\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.818264 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.818063 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.818264 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:28:22.818184 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-serving-cert: secret "maas-api-e2e-mt-e84500d1-serving-cert" not found Jul 15 19:28:22.818264 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:28:22.818265 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls podName:b339b9bf-92af-4c49-b585-d74ad18147a8 nodeName:}" failed. No retries permitted until 2026-07-15 19:28:23.318225129 +0000 UTC m=+1672.785199432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls") pod "maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" (UID: "b339b9bf-92af-4c49-b585-d74ad18147a8") : secret "maas-api-e2e-mt-e84500d1-serving-cert" not found Jul 15 19:28:22.818724 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.818704 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b339b9bf-92af-4c49-b585-d74ad18147a8-openshift-service-ca\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:22.827190 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:22.827165 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hzh9\" (UniqueName: \"kubernetes.io/projected/b339b9bf-92af-4c49-b585-d74ad18147a8-kube-api-access-4hzh9\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:23.323837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:23.323796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:23.326413 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:23.326389 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls\") pod \"maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:23.574452 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:23.574357 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:23.713765 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:23.713736 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8"] Jul 15 19:28:23.716439 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:28:23.716411 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb339b9bf_92af_4c49_b585_d74ad18147a8.slice/crio-1c1fec84b81804862f7098cc8783a4d3b8e0c9f4a73d167b406ce47e910406a9 WatchSource:0}: Error finding container 1c1fec84b81804862f7098cc8783a4d3b8e0c9f4a73d167b406ce47e910406a9: Status 404 returned error can't find the container with id 1c1fec84b81804862f7098cc8783a4d3b8e0c9f4a73d167b406ce47e910406a9 Jul 15 19:28:23.799461 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:23.799428 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" event={"ID":"b339b9bf-92af-4c49-b585-d74ad18147a8","Type":"ContainerStarted","Data":"1c1fec84b81804862f7098cc8783a4d3b8e0c9f4a73d167b406ce47e910406a9"} Jul 15 19:28:24.805214 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:24.805177 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" event={"ID":"b339b9bf-92af-4c49-b585-d74ad18147a8","Type":"ContainerStarted","Data":"10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5"} Jul 15 19:28:24.805633 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:24.805317 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:24.830630 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:24.830571 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" podStartSLOduration=2.38052284 podStartE2EDuration="2.830553504s" totalImagesPullingTime="450.030664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:24.828650538 +0000 UTC m=+1674.295624851" watchObservedRunningTime="2026-07-15 19:28:24.830553504 +0000 UTC m=+1674.297527816" Jul 15 19:28:30.815845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:30.815813 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:40.538166 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.538028 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8"] Jul 15 19:28:40.538811 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.538317 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" podUID="b339b9bf-92af-4c49-b585-d74ad18147a8" containerName="maas-api" containerID="cri-o://10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5" gracePeriod=30 Jul 15 19:28:40.788522 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.788469 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:40.878735 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.878700 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls\") pod \"b339b9bf-92af-4c49-b585-d74ad18147a8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " Jul 15 19:28:40.878911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.878793 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hzh9\" (UniqueName: \"kubernetes.io/projected/b339b9bf-92af-4c49-b585-d74ad18147a8-kube-api-access-4hzh9\") pod \"b339b9bf-92af-4c49-b585-d74ad18147a8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " Jul 15 19:28:40.878911 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.878835 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b339b9bf-92af-4c49-b585-d74ad18147a8-openshift-service-ca\") pod \"b339b9bf-92af-4c49-b585-d74ad18147a8\" (UID: \"b339b9bf-92af-4c49-b585-d74ad18147a8\") " Jul 15 19:28:40.879264 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.879207 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b339b9bf-92af-4c49-b585-d74ad18147a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b339b9bf-92af-4c49-b585-d74ad18147a8" (UID: "b339b9bf-92af-4c49-b585-d74ad18147a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:28:40.880977 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.880943 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b339b9bf-92af-4c49-b585-d74ad18147a8" (UID: "b339b9bf-92af-4c49-b585-d74ad18147a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:28:40.881110 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.881083 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b339b9bf-92af-4c49-b585-d74ad18147a8-kube-api-access-4hzh9" (OuterVolumeSpecName: "kube-api-access-4hzh9") pod "b339b9bf-92af-4c49-b585-d74ad18147a8" (UID: "b339b9bf-92af-4c49-b585-d74ad18147a8"). InnerVolumeSpecName "kube-api-access-4hzh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:28:40.884318 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.884292 2572 generic.go:356] "Generic (PLEG): container finished" podID="b339b9bf-92af-4c49-b585-d74ad18147a8" containerID="10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5" exitCode=0 Jul 15 19:28:40.884426 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.884362 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" event={"ID":"b339b9bf-92af-4c49-b585-d74ad18147a8","Type":"ContainerDied","Data":"10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5"} Jul 15 19:28:40.884426 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.884370 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" Jul 15 19:28:40.884426 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.884404 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8" event={"ID":"b339b9bf-92af-4c49-b585-d74ad18147a8","Type":"ContainerDied","Data":"1c1fec84b81804862f7098cc8783a4d3b8e0c9f4a73d167b406ce47e910406a9"} Jul 15 19:28:40.884588 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.884427 2572 scope.go:117] "RemoveContainer" containerID="10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5" Jul 15 19:28:40.898818 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.898799 2572 scope.go:117] "RemoveContainer" containerID="10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5" Jul 15 19:28:40.899114 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:28:40.899093 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5\": container with ID starting with 10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5 not found: ID does not exist" containerID="10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5" Jul 15 19:28:40.899161 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.899125 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5"} err="failed to get container status \"10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5\": rpc error: code = NotFound desc = could not find container \"10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5\": container with ID starting with 10ee9c28b47c0d5e74140b2c6e54f879659a636b3707ce545c5afb7c2c4caeb5 not found: ID does not exist" Jul 15 19:28:40.910977 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.910952 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8"] Jul 15 19:28:40.917145 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.917122 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e84500d1-bf597d46f-tnzb8"] Jul 15 19:28:40.980202 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.980169 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hzh9\" (UniqueName: \"kubernetes.io/projected/b339b9bf-92af-4c49-b585-d74ad18147a8-kube-api-access-4hzh9\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:28:40.980202 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.980199 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b339b9bf-92af-4c49-b585-d74ad18147a8-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:28:40.980403 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:40.980211 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b339b9bf-92af-4c49-b585-d74ad18147a8-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:28:41.034453 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:28:41.034420 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b339b9bf-92af-4c49-b585-d74ad18147a8" path="/var/lib/kubelet/pods/b339b9bf-92af-4c49-b585-d74ad18147a8/volumes" Jul 15 19:30:00.159943 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.159907 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2"] Jul 15 19:30:00.160435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.160247 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b339b9bf-92af-4c49-b585-d74ad18147a8" containerName="maas-api" Jul 15 19:30:00.160435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.160261 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b339b9bf-92af-4c49-b585-d74ad18147a8" containerName="maas-api" Jul 15 19:30:00.160435 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.160323 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b339b9bf-92af-4c49-b585-d74ad18147a8" containerName="maas-api" Jul 15 19:30:00.163202 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.163184 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" Jul 15 19:30:00.166138 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.166121 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jwf9b\"" Jul 15 19:30:00.172398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.172377 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2"] Jul 15 19:30:00.173802 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.173779 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" Jul 15 19:30:00.313682 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:00.313655 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2"] Jul 15 19:30:00.315677 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:30:00.315649 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b969d42_e293_4950_a9bc_88c2838a39ee.slice/crio-d86e5585c1c40fd66900ecf7400026c0d7d59efe1476b1737abddd701132c3df WatchSource:0}: Error finding container d86e5585c1c40fd66900ecf7400026c0d7d59efe1476b1737abddd701132c3df: Status 404 returned error can't find the container with id d86e5585c1c40fd66900ecf7400026c0d7d59efe1476b1737abddd701132c3df Jul 15 19:30:01.249002 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:01.248970 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" event={"ID":"7b969d42-e293-4950-a9bc-88c2838a39ee","Type":"ContainerStarted","Data":"dbddaccc276d80ad2f959caa90b434c6ec07e205b6e1bab467a8f2c93009c553"} Jul 15 19:30:01.249365 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:01.249028 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" event={"ID":"7b969d42-e293-4950-a9bc-88c2838a39ee","Type":"ContainerStarted","Data":"d86e5585c1c40fd66900ecf7400026c0d7d59efe1476b1737abddd701132c3df"} Jul 15 19:30:01.270115 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:01.270065 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" podStartSLOduration=1.270046032 podStartE2EDuration="1.270046032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:01.268213975 +0000 UTC m=+1770.735188283" watchObservedRunningTime="2026-07-15 19:30:01.270046032 +0000 UTC m=+1770.737020343" Jul 15 19:30:02.255698 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:02.255657 2572 generic.go:356] "Generic (PLEG): container finished" podID="7b969d42-e293-4950-a9bc-88c2838a39ee" containerID="dbddaccc276d80ad2f959caa90b434c6ec07e205b6e1bab467a8f2c93009c553" exitCode=0 Jul 15 19:30:02.256107 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:02.255743 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" event={"ID":"7b969d42-e293-4950-a9bc-88c2838a39ee","Type":"ContainerDied","Data":"dbddaccc276d80ad2f959caa90b434c6ec07e205b6e1bab467a8f2c93009c553"} Jul 15 19:30:03.390044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:03.390020 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" Jul 15 19:30:04.000387 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.000353 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg"] Jul 15 19:30:04.000862 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.000846 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b969d42-e293-4950-a9bc-88c2838a39ee" containerName="cleanup" Jul 15 19:30:04.000906 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.000866 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b969d42-e293-4950-a9bc-88c2838a39ee" containerName="cleanup" Jul 15 19:30:04.000986 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.000975 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b969d42-e293-4950-a9bc-88c2838a39ee" containerName="cleanup" Jul 15 19:30:04.004316 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.004293 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.007354 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.007329 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 19:30:04.007354 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.007352 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bdsfz\"" Jul 15 19:30:04.008201 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.008181 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4256e5-serving-cert\"" Jul 15 19:30:04.008490 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.008475 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:30:04.014306 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.014283 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg"] Jul 15 19:30:04.085358 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.085326 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b80ea5-956b-4605-8299-d2ec8c36f5c5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.085527 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.085370 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.085527 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.085396 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8wj8\" (UniqueName: \"kubernetes.io/projected/01b80ea5-956b-4605-8299-d2ec8c36f5c5-kube-api-access-l8wj8\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.186227 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.186196 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.186227 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.186248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8wj8\" (UniqueName: \"kubernetes.io/projected/01b80ea5-956b-4605-8299-d2ec8c36f5c5-kube-api-access-l8wj8\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.186491 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.186319 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b80ea5-956b-4605-8299-d2ec8c36f5c5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.186491 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:30:04.186355 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-serving-cert: secret "maas-api-e2e-inf-a-4256e5-serving-cert" not found Jul 15 19:30:04.186491 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:30:04.186433 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls podName:01b80ea5-956b-4605-8299-d2ec8c36f5c5 nodeName:}" failed. No retries permitted until 2026-07-15 19:30:04.686411711 +0000 UTC m=+1774.153386007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls") pod "maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" (UID: "01b80ea5-956b-4605-8299-d2ec8c36f5c5") : secret "maas-api-e2e-inf-a-4256e5-serving-cert" not found Jul 15 19:30:04.186884 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.186867 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b80ea5-956b-4605-8299-d2ec8c36f5c5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.202549 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.202519 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8wj8\" (UniqueName: \"kubernetes.io/projected/01b80ea5-956b-4605-8299-d2ec8c36f5c5-kube-api-access-l8wj8\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.266354 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.266274 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" Jul 15 19:30:04.266354 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.266291 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735730-zcfn2" event={"ID":"7b969d42-e293-4950-a9bc-88c2838a39ee","Type":"ContainerDied","Data":"d86e5585c1c40fd66900ecf7400026c0d7d59efe1476b1737abddd701132c3df"} Jul 15 19:30:04.266354 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.266334 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d86e5585c1c40fd66900ecf7400026c0d7d59efe1476b1737abddd701132c3df" Jul 15 19:30:04.691125 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.691093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.693659 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.693633 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:04.915538 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:04.915502 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:05.062947 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:05.062913 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg"] Jul 15 19:30:05.068654 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:30:05.068622 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b80ea5_956b_4605_8299_d2ec8c36f5c5.slice/crio-5f3b53f1d59113820969b85a2e1b578fd0bc9e667d925a13c5ffc8d082ad623e WatchSource:0}: Error finding container 5f3b53f1d59113820969b85a2e1b578fd0bc9e667d925a13c5ffc8d082ad623e: Status 404 returned error can't find the container with id 5f3b53f1d59113820969b85a2e1b578fd0bc9e667d925a13c5ffc8d082ad623e Jul 15 19:30:05.272760 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:05.272675 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" event={"ID":"01b80ea5-956b-4605-8299-d2ec8c36f5c5","Type":"ContainerStarted","Data":"5f3b53f1d59113820969b85a2e1b578fd0bc9e667d925a13c5ffc8d082ad623e"} Jul 15 19:30:06.283697 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:06.283654 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" event={"ID":"01b80ea5-956b-4605-8299-d2ec8c36f5c5","Type":"ContainerStarted","Data":"cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f"} Jul 15 19:30:06.284542 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:06.284470 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:06.306626 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:06.306572 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" podStartSLOduration=2.882234034 podStartE2EDuration="3.30655865s" totalImagesPullingTime="424.324616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:06.304392754 +0000 UTC m=+1775.771367068" watchObservedRunningTime="2026-07-15 19:30:06.30655865 +0000 UTC m=+1775.773532962" Jul 15 19:30:12.153265 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.153209 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm"] Jul 15 19:30:12.156781 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.156761 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.160596 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.160562 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-381b50-serving-cert\"" Jul 15 19:30:12.172669 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.172641 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm"] Jul 15 19:30:12.252748 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.252716 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aae5bed-3819-4b74-9fbb-caae210b3ad1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.252921 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.252773 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aae5bed-3819-4b74-9fbb-caae210b3ad1-maas-api-tls\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.252921 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.252837 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zf9t\" (UniqueName: \"kubernetes.io/projected/8aae5bed-3819-4b74-9fbb-caae210b3ad1-kube-api-access-6zf9t\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.293675 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.293645 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:30:12.354185 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.354154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zf9t\" (UniqueName: \"kubernetes.io/projected/8aae5bed-3819-4b74-9fbb-caae210b3ad1-kube-api-access-6zf9t\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.354390 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.354264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aae5bed-3819-4b74-9fbb-caae210b3ad1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.354390 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.354324 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aae5bed-3819-4b74-9fbb-caae210b3ad1-maas-api-tls\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.355020 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.354993 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aae5bed-3819-4b74-9fbb-caae210b3ad1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.356742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.356716 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aae5bed-3819-4b74-9fbb-caae210b3ad1-maas-api-tls\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.372717 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.372690 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zf9t\" (UniqueName: \"kubernetes.io/projected/8aae5bed-3819-4b74-9fbb-caae210b3ad1-kube-api-access-6zf9t\") pod \"maas-api-e2e-inf-b-381b50-64755b5db6-f88bm\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.467584 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.467489 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:12.619129 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:12.619093 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm"] Jul 15 19:30:12.620906 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:30:12.620877 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aae5bed_3819_4b74_9fbb_caae210b3ad1.slice/crio-2043af6dae7636096811bc4cde20c6ed3cd7a90bbf4a49921fd9330b7866de0a WatchSource:0}: Error finding container 2043af6dae7636096811bc4cde20c6ed3cd7a90bbf4a49921fd9330b7866de0a: Status 404 returned error can't find the container with id 2043af6dae7636096811bc4cde20c6ed3cd7a90bbf4a49921fd9330b7866de0a Jul 15 19:30:13.316877 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:13.316783 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" event={"ID":"8aae5bed-3819-4b74-9fbb-caae210b3ad1","Type":"ContainerStarted","Data":"9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c"} Jul 15 19:30:13.316877 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:13.316849 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" event={"ID":"8aae5bed-3819-4b74-9fbb-caae210b3ad1","Type":"ContainerStarted","Data":"2043af6dae7636096811bc4cde20c6ed3cd7a90bbf4a49921fd9330b7866de0a"} Jul 15 19:30:13.317304 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:13.316962 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:13.338670 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:13.338626 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" podStartSLOduration=0.938569014 podStartE2EDuration="1.338609512s" totalImagesPullingTime="400.040498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:13.336204501 +0000 UTC m=+1782.803178814" watchObservedRunningTime="2026-07-15 19:30:13.338609512 +0000 UTC m=+1782.805583857" Jul 15 19:30:19.328403 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:19.328364 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:30:31.117857 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:31.117821 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:30:31.126135 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:31.126107 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:30:31.126657 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:31.126629 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:30:31.135253 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:30:31.135208 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:31:59.151089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.150998 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg"] Jul 15 19:31:59.153484 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.151279 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" podUID="01b80ea5-956b-4605-8299-d2ec8c36f5c5" containerName="maas-api" containerID="cri-o://cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f" gracePeriod=30 Jul 15 19:31:59.406496 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.406436 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:31:59.455408 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.455374 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls\") pod \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " Jul 15 19:31:59.455558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.455450 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8wj8\" (UniqueName: \"kubernetes.io/projected/01b80ea5-956b-4605-8299-d2ec8c36f5c5-kube-api-access-l8wj8\") pod \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " Jul 15 19:31:59.455558 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.455483 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b80ea5-956b-4605-8299-d2ec8c36f5c5-openshift-service-ca\") pod \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\" (UID: \"01b80ea5-956b-4605-8299-d2ec8c36f5c5\") " Jul 15 19:31:59.455902 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.455871 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b80ea5-956b-4605-8299-d2ec8c36f5c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01b80ea5-956b-4605-8299-d2ec8c36f5c5" (UID: "01b80ea5-956b-4605-8299-d2ec8c36f5c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:31:59.457418 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.457391 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01b80ea5-956b-4605-8299-d2ec8c36f5c5" (UID: "01b80ea5-956b-4605-8299-d2ec8c36f5c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:31:59.457563 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.457547 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b80ea5-956b-4605-8299-d2ec8c36f5c5-kube-api-access-l8wj8" (OuterVolumeSpecName: "kube-api-access-l8wj8") pod "01b80ea5-956b-4605-8299-d2ec8c36f5c5" (UID: "01b80ea5-956b-4605-8299-d2ec8c36f5c5"). InnerVolumeSpecName "kube-api-access-l8wj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:31:59.557194 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.557152 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b80ea5-956b-4605-8299-d2ec8c36f5c5-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:31:59.557194 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.557185 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b80ea5-956b-4605-8299-d2ec8c36f5c5-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:31:59.557194 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.557195 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8wj8\" (UniqueName: \"kubernetes.io/projected/01b80ea5-956b-4605-8299-d2ec8c36f5c5-kube-api-access-l8wj8\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:31:59.824026 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.823940 2572 generic.go:356] "Generic (PLEG): container finished" podID="01b80ea5-956b-4605-8299-d2ec8c36f5c5" containerID="cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f" exitCode=0 Jul 15 19:31:59.824026 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.823984 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" Jul 15 19:31:59.824304 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.824031 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" event={"ID":"01b80ea5-956b-4605-8299-d2ec8c36f5c5","Type":"ContainerDied","Data":"cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f"} Jul 15 19:31:59.824304 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.824091 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg" event={"ID":"01b80ea5-956b-4605-8299-d2ec8c36f5c5","Type":"ContainerDied","Data":"5f3b53f1d59113820969b85a2e1b578fd0bc9e667d925a13c5ffc8d082ad623e"} Jul 15 19:31:59.824304 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.824121 2572 scope.go:117] "RemoveContainer" containerID="cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f" Jul 15 19:31:59.833150 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.833133 2572 scope.go:117] "RemoveContainer" containerID="cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f" Jul 15 19:31:59.833508 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:31:59.833482 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f\": container with ID starting with cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f not found: ID does not exist" containerID="cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f" Jul 15 19:31:59.833629 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.833519 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f"} err="failed to get container status \"cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f\": rpc error: code = NotFound desc = could not find container \"cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f\": container with ID starting with cdeda1fa0ac6e616a6f4945b6115591ab6d0fd36afdeb0f5f5aabfa2a881a58f not found: ID does not exist" Jul 15 19:31:59.848775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.848746 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg"] Jul 15 19:31:59.853058 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:31:59.853036 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4256e5-6dfd569599-47kdg"] Jul 15 19:32:01.034586 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:01.034546 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b80ea5-956b-4605-8299-d2ec8c36f5c5" path="/var/lib/kubelet/pods/01b80ea5-956b-4605-8299-d2ec8c36f5c5/volumes" Jul 15 19:32:11.594149 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.594115 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm"] Jul 15 19:32:11.594726 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.594414 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" podUID="8aae5bed-3819-4b74-9fbb-caae210b3ad1" containerName="maas-api" containerID="cri-o://9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c" gracePeriod=30 Jul 15 19:32:11.846391 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.846368 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:32:11.884492 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.884460 2572 generic.go:356] "Generic (PLEG): container finished" podID="8aae5bed-3819-4b74-9fbb-caae210b3ad1" containerID="9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c" exitCode=0 Jul 15 19:32:11.884662 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.884510 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" Jul 15 19:32:11.884662 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.884514 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" event={"ID":"8aae5bed-3819-4b74-9fbb-caae210b3ad1","Type":"ContainerDied","Data":"9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c"} Jul 15 19:32:11.884662 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.884641 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm" event={"ID":"8aae5bed-3819-4b74-9fbb-caae210b3ad1","Type":"ContainerDied","Data":"2043af6dae7636096811bc4cde20c6ed3cd7a90bbf4a49921fd9330b7866de0a"} Jul 15 19:32:11.884662 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.884664 2572 scope.go:117] "RemoveContainer" containerID="9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c" Jul 15 19:32:11.893446 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.893396 2572 scope.go:117] "RemoveContainer" containerID="9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c" Jul 15 19:32:11.893670 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:32:11.893652 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c\": container with ID starting with 9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c not found: ID does not exist" containerID="9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c" Jul 15 19:32:11.893733 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.893679 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c"} err="failed to get container status \"9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c\": rpc error: code = NotFound desc = could not find container \"9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c\": container with ID starting with 9d8694ab26b7584c2841d9f42f2062f50d594ef57458b6d35e53683559b63b6c not found: ID does not exist" Jul 15 19:32:11.956657 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.956635 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zf9t\" (UniqueName: \"kubernetes.io/projected/8aae5bed-3819-4b74-9fbb-caae210b3ad1-kube-api-access-6zf9t\") pod \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " Jul 15 19:32:11.956796 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.956755 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aae5bed-3819-4b74-9fbb-caae210b3ad1-maas-api-tls\") pod \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " Jul 15 19:32:11.956862 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.956812 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aae5bed-3819-4b74-9fbb-caae210b3ad1-openshift-service-ca\") pod \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\" (UID: \"8aae5bed-3819-4b74-9fbb-caae210b3ad1\") " Jul 15 19:32:11.957163 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.957138 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aae5bed-3819-4b74-9fbb-caae210b3ad1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8aae5bed-3819-4b74-9fbb-caae210b3ad1" (UID: "8aae5bed-3819-4b74-9fbb-caae210b3ad1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:32:11.958784 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.958760 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aae5bed-3819-4b74-9fbb-caae210b3ad1-kube-api-access-6zf9t" (OuterVolumeSpecName: "kube-api-access-6zf9t") pod "8aae5bed-3819-4b74-9fbb-caae210b3ad1" (UID: "8aae5bed-3819-4b74-9fbb-caae210b3ad1"). InnerVolumeSpecName "kube-api-access-6zf9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:11.958866 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:11.958762 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aae5bed-3819-4b74-9fbb-caae210b3ad1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8aae5bed-3819-4b74-9fbb-caae210b3ad1" (UID: "8aae5bed-3819-4b74-9fbb-caae210b3ad1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:32:12.058409 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:12.058378 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aae5bed-3819-4b74-9fbb-caae210b3ad1-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:32:12.058409 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:12.058405 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aae5bed-3819-4b74-9fbb-caae210b3ad1-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:32:12.058568 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:12.058420 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zf9t\" (UniqueName: \"kubernetes.io/projected/8aae5bed-3819-4b74-9fbb-caae210b3ad1-kube-api-access-6zf9t\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:32:12.208198 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:12.208171 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm"] Jul 15 19:32:12.212508 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:12.212481 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-381b50-64755b5db6-f88bm"] Jul 15 19:32:13.033467 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:13.033431 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aae5bed-3819-4b74-9fbb-caae210b3ad1" path="/var/lib/kubelet/pods/8aae5bed-3819-4b74-9fbb-caae210b3ad1/volumes" Jul 15 19:32:21.948438 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.948401 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8"] Jul 15 19:32:21.948887 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.948870 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b80ea5-956b-4605-8299-d2ec8c36f5c5" containerName="maas-api" Jul 15 19:32:21.948940 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.948890 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b80ea5-956b-4605-8299-d2ec8c36f5c5" containerName="maas-api" Jul 15 19:32:21.948940 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.948930 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aae5bed-3819-4b74-9fbb-caae210b3ad1" containerName="maas-api" Jul 15 19:32:21.949013 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.948938 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aae5bed-3819-4b74-9fbb-caae210b3ad1" containerName="maas-api" Jul 15 19:32:21.949013 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.948998 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b80ea5-956b-4605-8299-d2ec8c36f5c5" containerName="maas-api" Jul 15 19:32:21.949013 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.949007 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aae5bed-3819-4b74-9fbb-caae210b3ad1" containerName="maas-api" Jul 15 19:32:21.953632 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.953601 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:21.956357 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.956330 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4293f0-serving-cert\"" Jul 15 19:32:21.956478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.956355 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:32:21.957524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.957508 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bdsfz\"" Jul 15 19:32:21.957596 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.957520 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 19:32:21.962083 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:21.962059 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8"] Jul 15 19:32:22.051076 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.051040 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.051292 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.051089 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltwcc\" (UniqueName: \"kubernetes.io/projected/3794870a-f266-48c7-97f1-bf5acbe9f91c-kube-api-access-ltwcc\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.051292 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.051127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3794870a-f266-48c7-97f1-bf5acbe9f91c-openshift-service-ca\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.151900 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.151867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.152084 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.151914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltwcc\" (UniqueName: \"kubernetes.io/projected/3794870a-f266-48c7-97f1-bf5acbe9f91c-kube-api-access-ltwcc\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.152084 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.151952 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3794870a-f266-48c7-97f1-bf5acbe9f91c-openshift-service-ca\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.152084 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:32:22.152055 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-serving-cert: secret "maas-api-e2e-api-a-4293f0-serving-cert" not found Jul 15 19:32:22.152280 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:32:22.152130 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls podName:3794870a-f266-48c7-97f1-bf5acbe9f91c nodeName:}" failed. No retries permitted until 2026-07-15 19:32:22.652111076 +0000 UTC m=+1912.119085381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls") pod "maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" (UID: "3794870a-f266-48c7-97f1-bf5acbe9f91c") : secret "maas-api-e2e-api-a-4293f0-serving-cert" not found Jul 15 19:32:22.152753 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.152731 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3794870a-f266-48c7-97f1-bf5acbe9f91c-openshift-service-ca\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.163044 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.163025 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltwcc\" (UniqueName: \"kubernetes.io/projected/3794870a-f266-48c7-97f1-bf5acbe9f91c-kube-api-access-ltwcc\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.656309 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.656272 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.658827 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.658800 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls\") pod \"maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:22.865284 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:22.865223 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:23.001755 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:23.001728 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8"] Jul 15 19:32:23.004082 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:32:23.004055 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3794870a_f266_48c7_97f1_bf5acbe9f91c.slice/crio-f555384b5b3cb98cd5e3ffdf67c385574024751119f03637e54b34661042ca0b WatchSource:0}: Error finding container f555384b5b3cb98cd5e3ffdf67c385574024751119f03637e54b34661042ca0b: Status 404 returned error can't find the container with id f555384b5b3cb98cd5e3ffdf67c385574024751119f03637e54b34661042ca0b Jul 15 19:32:23.005775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:23.005758 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:32:23.944159 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:23.944114 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" event={"ID":"3794870a-f266-48c7-97f1-bf5acbe9f91c","Type":"ContainerStarted","Data":"90fc3713f0cf9fda877be8a88343238a42fe68842531353f27b51d4ac669fda5"} Jul 15 19:32:23.944353 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:23.944208 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:23.944353 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:23.944225 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" event={"ID":"3794870a-f266-48c7-97f1-bf5acbe9f91c","Type":"ContainerStarted","Data":"f555384b5b3cb98cd5e3ffdf67c385574024751119f03637e54b34661042ca0b"} Jul 15 19:32:23.962524 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:23.962476 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" podStartSLOduration=2.2234170779999998 podStartE2EDuration="2.962459631s" totalImagesPullingTime="739.042553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:23.960096918 +0000 UTC m=+1913.427071229" watchObservedRunningTime="2026-07-15 19:32:23.962459631 +0000 UTC m=+1913.429433942" Jul 15 19:32:29.859154 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:29.859117 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr"] Jul 15 19:32:29.863431 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:29.863406 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:29.867005 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:29.866977 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a115a3-serving-cert\"" Jul 15 19:32:29.876395 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:29.876371 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr"] Jul 15 19:32:29.954981 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:29.954951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:30.020425 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.020394 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mz5b\" (UniqueName: \"kubernetes.io/projected/c02bbee2-f9da-4a97-8a42-963b3809d118-kube-api-access-7mz5b\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.020608 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.020437 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c02bbee2-f9da-4a97-8a42-963b3809d118-openshift-service-ca\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.020608 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.020472 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.121222 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.121137 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mz5b\" (UniqueName: \"kubernetes.io/projected/c02bbee2-f9da-4a97-8a42-963b3809d118-kube-api-access-7mz5b\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.121222 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.121196 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c02bbee2-f9da-4a97-8a42-963b3809d118-openshift-service-ca\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.121222 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.121225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.121563 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:32:30.121362 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-serving-cert: secret "maas-api-e2e-api-b-a115a3-serving-cert" not found Jul 15 19:32:30.121563 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:32:30.121437 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls podName:c02bbee2-f9da-4a97-8a42-963b3809d118 nodeName:}" failed. No retries permitted until 2026-07-15 19:32:30.621422242 +0000 UTC m=+1920.088396532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls") pod "maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" (UID: "c02bbee2-f9da-4a97-8a42-963b3809d118") : secret "maas-api-e2e-api-b-a115a3-serving-cert" not found Jul 15 19:32:30.121912 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.121888 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c02bbee2-f9da-4a97-8a42-963b3809d118-openshift-service-ca\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.130045 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.130016 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mz5b\" (UniqueName: \"kubernetes.io/projected/c02bbee2-f9da-4a97-8a42-963b3809d118-kube-api-access-7mz5b\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.626853 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.626810 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.635100 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.630414 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls\") pod \"maas-api-e2e-api-b-a115a3-6cc4959974-92vbr\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.776586 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.776543 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:30.910767 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.910733 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr"] Jul 15 19:32:30.911640 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:32:30.911616 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02bbee2_f9da_4a97_8a42_963b3809d118.slice/crio-b70a089d0f05bd09c5c0ad9838e1efe87ee4b8babc3111524a8e6bcf5ae17ca6 WatchSource:0}: Error finding container b70a089d0f05bd09c5c0ad9838e1efe87ee4b8babc3111524a8e6bcf5ae17ca6: Status 404 returned error can't find the container with id b70a089d0f05bd09c5c0ad9838e1efe87ee4b8babc3111524a8e6bcf5ae17ca6 Jul 15 19:32:30.979916 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:30.979885 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" event={"ID":"c02bbee2-f9da-4a97-8a42-963b3809d118","Type":"ContainerStarted","Data":"b70a089d0f05bd09c5c0ad9838e1efe87ee4b8babc3111524a8e6bcf5ae17ca6"} Jul 15 19:32:31.985852 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:31.985807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" event={"ID":"c02bbee2-f9da-4a97-8a42-963b3809d118","Type":"ContainerStarted","Data":"70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855"} Jul 15 19:32:31.986218 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:31.985951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:32.003352 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:32.003298 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" podStartSLOduration=2.505089526 podStartE2EDuration="3.003281849s" totalImagesPullingTime="498.192323ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:32.002326971 +0000 UTC m=+1921.469301283" watchObservedRunningTime="2026-07-15 19:32:32.003281849 +0000 UTC m=+1921.470256158" Jul 15 19:32:37.995360 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:37.995328 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:32:48.948502 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:48.948469 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8"] Jul 15 19:32:48.948993 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:48.948703 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" podUID="3794870a-f266-48c7-97f1-bf5acbe9f91c" containerName="maas-api" containerID="cri-o://90fc3713f0cf9fda877be8a88343238a42fe68842531353f27b51d4ac669fda5" gracePeriod=30 Jul 15 19:32:49.071970 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.071939 2572 generic.go:356] "Generic (PLEG): container finished" podID="3794870a-f266-48c7-97f1-bf5acbe9f91c" containerID="90fc3713f0cf9fda877be8a88343238a42fe68842531353f27b51d4ac669fda5" exitCode=0 Jul 15 19:32:49.072089 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.071973 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" event={"ID":"3794870a-f266-48c7-97f1-bf5acbe9f91c","Type":"ContainerDied","Data":"90fc3713f0cf9fda877be8a88343238a42fe68842531353f27b51d4ac669fda5"} Jul 15 19:32:49.196000 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.195972 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:49.296611 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.296529 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3794870a-f266-48c7-97f1-bf5acbe9f91c-openshift-service-ca\") pod \"3794870a-f266-48c7-97f1-bf5acbe9f91c\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " Jul 15 19:32:49.296611 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.296577 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls\") pod \"3794870a-f266-48c7-97f1-bf5acbe9f91c\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " Jul 15 19:32:49.296835 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.296651 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltwcc\" (UniqueName: \"kubernetes.io/projected/3794870a-f266-48c7-97f1-bf5acbe9f91c-kube-api-access-ltwcc\") pod \"3794870a-f266-48c7-97f1-bf5acbe9f91c\" (UID: \"3794870a-f266-48c7-97f1-bf5acbe9f91c\") " Jul 15 19:32:49.296933 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.296905 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3794870a-f266-48c7-97f1-bf5acbe9f91c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3794870a-f266-48c7-97f1-bf5acbe9f91c" (UID: "3794870a-f266-48c7-97f1-bf5acbe9f91c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:32:49.298805 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.298778 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3794870a-f266-48c7-97f1-bf5acbe9f91c-kube-api-access-ltwcc" (OuterVolumeSpecName: "kube-api-access-ltwcc") pod "3794870a-f266-48c7-97f1-bf5acbe9f91c" (UID: "3794870a-f266-48c7-97f1-bf5acbe9f91c"). InnerVolumeSpecName "kube-api-access-ltwcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:49.298805 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.298800 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3794870a-f266-48c7-97f1-bf5acbe9f91c" (UID: "3794870a-f266-48c7-97f1-bf5acbe9f91c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:32:49.397959 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.397922 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3794870a-f266-48c7-97f1-bf5acbe9f91c-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:32:49.397959 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.397953 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3794870a-f266-48c7-97f1-bf5acbe9f91c-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:32:49.397959 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:49.397962 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltwcc\" (UniqueName: \"kubernetes.io/projected/3794870a-f266-48c7-97f1-bf5acbe9f91c-kube-api-access-ltwcc\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:32:50.078182 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:50.078144 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" Jul 15 19:32:50.078598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:50.078178 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8" event={"ID":"3794870a-f266-48c7-97f1-bf5acbe9f91c","Type":"ContainerDied","Data":"f555384b5b3cb98cd5e3ffdf67c385574024751119f03637e54b34661042ca0b"} Jul 15 19:32:50.078598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:50.078254 2572 scope.go:117] "RemoveContainer" containerID="90fc3713f0cf9fda877be8a88343238a42fe68842531353f27b51d4ac669fda5" Jul 15 19:32:50.102399 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:50.102369 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8"] Jul 15 19:32:50.104282 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:50.104258 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4293f0-5986889cf7-c9sq8"] Jul 15 19:32:51.040169 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:32:51.040131 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3794870a-f266-48c7-97f1-bf5acbe9f91c" path="/var/lib/kubelet/pods/3794870a-f266-48c7-97f1-bf5acbe9f91c/volumes" Jul 15 19:33:00.878822 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:00.878779 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr"] Jul 15 19:33:00.879342 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:00.879039 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" podUID="c02bbee2-f9da-4a97-8a42-963b3809d118" containerName="maas-api" containerID="cri-o://70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855" gracePeriod=30 Jul 15 19:33:01.125215 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.125190 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:33:01.131196 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.131136 2572 generic.go:356] "Generic (PLEG): container finished" podID="c02bbee2-f9da-4a97-8a42-963b3809d118" containerID="70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855" exitCode=0 Jul 15 19:33:01.131196 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.131177 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" Jul 15 19:33:01.131398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.131213 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" event={"ID":"c02bbee2-f9da-4a97-8a42-963b3809d118","Type":"ContainerDied","Data":"70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855"} Jul 15 19:33:01.131398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.131279 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr" event={"ID":"c02bbee2-f9da-4a97-8a42-963b3809d118","Type":"ContainerDied","Data":"b70a089d0f05bd09c5c0ad9838e1efe87ee4b8babc3111524a8e6bcf5ae17ca6"} Jul 15 19:33:01.131398 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.131303 2572 scope.go:117] "RemoveContainer" containerID="70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855" Jul 15 19:33:01.139801 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.139776 2572 scope.go:117] "RemoveContainer" containerID="70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855" Jul 15 19:33:01.140121 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:33:01.140099 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855\": container with ID starting with 70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855 not found: ID does not exist" containerID="70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855" Jul 15 19:33:01.140221 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.140134 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855"} err="failed to get container status \"70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855\": rpc error: code = NotFound desc = could not find container \"70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855\": container with ID starting with 70ca41608165ce471a9a146d47f41f6a046b9511b6a9df9da3bb7bd69b448855 not found: ID does not exist" Jul 15 19:33:01.198696 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.198664 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls\") pod \"c02bbee2-f9da-4a97-8a42-963b3809d118\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " Jul 15 19:33:01.198877 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.198742 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mz5b\" (UniqueName: \"kubernetes.io/projected/c02bbee2-f9da-4a97-8a42-963b3809d118-kube-api-access-7mz5b\") pod \"c02bbee2-f9da-4a97-8a42-963b3809d118\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " Jul 15 19:33:01.198877 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.198831 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c02bbee2-f9da-4a97-8a42-963b3809d118-openshift-service-ca\") pod \"c02bbee2-f9da-4a97-8a42-963b3809d118\" (UID: \"c02bbee2-f9da-4a97-8a42-963b3809d118\") " Jul 15 19:33:01.199182 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.199160 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02bbee2-f9da-4a97-8a42-963b3809d118-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c02bbee2-f9da-4a97-8a42-963b3809d118" (UID: "c02bbee2-f9da-4a97-8a42-963b3809d118"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:33:01.200819 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.200792 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02bbee2-f9da-4a97-8a42-963b3809d118-kube-api-access-7mz5b" (OuterVolumeSpecName: "kube-api-access-7mz5b") pod "c02bbee2-f9da-4a97-8a42-963b3809d118" (UID: "c02bbee2-f9da-4a97-8a42-963b3809d118"). InnerVolumeSpecName "kube-api-access-7mz5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:33:01.200819 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.200803 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c02bbee2-f9da-4a97-8a42-963b3809d118" (UID: "c02bbee2-f9da-4a97-8a42-963b3809d118"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:33:01.299995 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.299959 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c02bbee2-f9da-4a97-8a42-963b3809d118-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:33:01.299995 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.299990 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c02bbee2-f9da-4a97-8a42-963b3809d118-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:33:01.299995 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.299999 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mz5b\" (UniqueName: \"kubernetes.io/projected/c02bbee2-f9da-4a97-8a42-963b3809d118-kube-api-access-7mz5b\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:33:01.456063 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.456022 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr"] Jul 15 19:33:01.460389 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:01.460361 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a115a3-6cc4959974-92vbr"] Jul 15 19:33:03.033118 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:03.033081 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02bbee2-f9da-4a97-8a42-963b3809d118" path="/var/lib/kubelet/pods/c02bbee2-f9da-4a97-8a42-963b3809d118/volumes" Jul 15 19:33:11.156768 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.156727 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx"] Jul 15 19:33:11.157273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.157054 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c02bbee2-f9da-4a97-8a42-963b3809d118" containerName="maas-api" Jul 15 19:33:11.157273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.157065 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02bbee2-f9da-4a97-8a42-963b3809d118" containerName="maas-api" Jul 15 19:33:11.157273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.157078 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3794870a-f266-48c7-97f1-bf5acbe9f91c" containerName="maas-api" Jul 15 19:33:11.157273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.157083 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3794870a-f266-48c7-97f1-bf5acbe9f91c" containerName="maas-api" Jul 15 19:33:11.157273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.157134 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c02bbee2-f9da-4a97-8a42-963b3809d118" containerName="maas-api" Jul 15 19:33:11.157273 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.157145 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3794870a-f266-48c7-97f1-bf5acbe9f91c" containerName="maas-api" Jul 15 19:33:11.160328 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.160305 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.163122 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.163102 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-4a4ed2-serving-cert\"" Jul 15 19:33:11.163384 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.163365 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:33:11.163487 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.163425 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bdsfz\"" Jul 15 19:33:11.164297 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.164277 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 19:33:11.173267 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.173226 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx"] Jul 15 19:33:11.282706 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.282669 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.282895 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.282734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47271a67-32ae-48a5-a2dd-9da2b6683b8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.282895 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.282851 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9hcw\" (UniqueName: \"kubernetes.io/projected/47271a67-32ae-48a5-a2dd-9da2b6683b8d-kube-api-access-t9hcw\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.384061 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.384026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47271a67-32ae-48a5-a2dd-9da2b6683b8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.384206 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.384073 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9hcw\" (UniqueName: \"kubernetes.io/projected/47271a67-32ae-48a5-a2dd-9da2b6683b8d-kube-api-access-t9hcw\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.384206 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.384108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.384206 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:33:11.384195 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-serving-cert: secret "maas-api-e2e-shared-a-4a4ed2-serving-cert" not found Jul 15 19:33:11.384415 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:33:11.384307 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls podName:47271a67-32ae-48a5-a2dd-9da2b6683b8d nodeName:}" failed. No retries permitted until 2026-07-15 19:33:11.884289504 +0000 UTC m=+1961.351263797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls") pod "maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" (UID: "47271a67-32ae-48a5-a2dd-9da2b6683b8d") : secret "maas-api-e2e-shared-a-4a4ed2-serving-cert" not found Jul 15 19:33:11.384688 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.384666 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47271a67-32ae-48a5-a2dd-9da2b6683b8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.393169 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.393142 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9hcw\" (UniqueName: \"kubernetes.io/projected/47271a67-32ae-48a5-a2dd-9da2b6683b8d-kube-api-access-t9hcw\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.888812 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.888776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:11.891249 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:11.891208 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:12.074017 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:12.073974 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:12.233545 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:12.233515 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx"] Jul 15 19:33:12.235270 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:33:12.235228 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47271a67_32ae_48a5_a2dd_9da2b6683b8d.slice/crio-437bfe9f149d139eb258a7d77b863ee42abcc88cf77b6dc8e74c0f7294b99b91 WatchSource:0}: Error finding container 437bfe9f149d139eb258a7d77b863ee42abcc88cf77b6dc8e74c0f7294b99b91: Status 404 returned error can't find the container with id 437bfe9f149d139eb258a7d77b863ee42abcc88cf77b6dc8e74c0f7294b99b91 Jul 15 19:33:13.207725 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:13.207691 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" event={"ID":"47271a67-32ae-48a5-a2dd-9da2b6683b8d","Type":"ContainerStarted","Data":"d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9"} Jul 15 19:33:13.207921 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:13.207776 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:13.207921 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:13.207794 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" event={"ID":"47271a67-32ae-48a5-a2dd-9da2b6683b8d","Type":"ContainerStarted","Data":"437bfe9f149d139eb258a7d77b863ee42abcc88cf77b6dc8e74c0f7294b99b91"} Jul 15 19:33:13.267941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:13.267880 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" podStartSLOduration=1.864764714 podStartE2EDuration="2.267862882s" totalImagesPullingTime="403.098168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:33:13.266309075 +0000 UTC m=+1962.733283387" watchObservedRunningTime="2026-07-15 19:33:13.267862882 +0000 UTC m=+1962.734837193" Jul 15 19:33:19.096305 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.096266 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9"] Jul 15 19:33:19.099872 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.099850 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.102730 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.102703 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f83827-serving-cert\"" Jul 15 19:33:19.110557 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.110527 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9"] Jul 15 19:33:19.218066 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.218037 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:33:19.251775 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.251738 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba022e91-d3ac-4241-90bd-494af8e3fec0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.251974 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.251847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba022e91-d3ac-4241-90bd-494af8e3fec0-maas-api-tls\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.252051 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.251972 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cth2h\" (UniqueName: \"kubernetes.io/projected/ba022e91-d3ac-4241-90bd-494af8e3fec0-kube-api-access-cth2h\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.353205 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.353101 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cth2h\" (UniqueName: \"kubernetes.io/projected/ba022e91-d3ac-4241-90bd-494af8e3fec0-kube-api-access-cth2h\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.353205 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.353186 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba022e91-d3ac-4241-90bd-494af8e3fec0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.353478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.353320 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba022e91-d3ac-4241-90bd-494af8e3fec0-maas-api-tls\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.353880 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.353855 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba022e91-d3ac-4241-90bd-494af8e3fec0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.355786 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.355766 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba022e91-d3ac-4241-90bd-494af8e3fec0-maas-api-tls\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.364767 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.364744 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cth2h\" (UniqueName: \"kubernetes.io/projected/ba022e91-d3ac-4241-90bd-494af8e3fec0-kube-api-access-cth2h\") pod \"maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.412208 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.412167 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:19.553133 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:19.553100 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9"] Jul 15 19:33:19.555262 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:33:19.555210 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba022e91_d3ac_4241_90bd_494af8e3fec0.slice/crio-1113377e2a0f1f56906dc943d5682f5f606e10349f2511ba2be08043610c9f37 WatchSource:0}: Error finding container 1113377e2a0f1f56906dc943d5682f5f606e10349f2511ba2be08043610c9f37: Status 404 returned error can't find the container with id 1113377e2a0f1f56906dc943d5682f5f606e10349f2511ba2be08043610c9f37 Jul 15 19:33:20.243137 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:20.243036 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" event={"ID":"ba022e91-d3ac-4241-90bd-494af8e3fec0","Type":"ContainerStarted","Data":"bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b"} Jul 15 19:33:20.243137 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:20.243102 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:33:20.243137 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:20.243119 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" event={"ID":"ba022e91-d3ac-4241-90bd-494af8e3fec0","Type":"ContainerStarted","Data":"1113377e2a0f1f56906dc943d5682f5f606e10349f2511ba2be08043610c9f37"} Jul 15 19:33:20.268759 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:20.268684 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" podStartSLOduration=0.85612255 podStartE2EDuration="1.268665751s" totalImagesPullingTime="412.543201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:33:20.267645358 +0000 UTC m=+1969.734619670" watchObservedRunningTime="2026-07-15 19:33:20.268665751 +0000 UTC m=+1969.735640062" Jul 15 19:33:26.253992 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:33:26.253910 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:35:31.152559 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:35:31.152486 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:35:31.159439 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:35:31.159416 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:35:31.160793 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:35:31.160772 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:35:31.167845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:35:31.167824 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:36:23.132013 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.131974 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj"] Jul 15 19:36:23.135715 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.135693 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.140941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.140839 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a4ed2\"/\"default-dockercfg-zqnxm\"" Jul 15 19:36:23.140941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.140853 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a4ed2\"/\"kube-root-ca.crt\"" Jul 15 19:36:23.140941 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.140922 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a4ed2\"/\"openshift-service-ca.crt\"" Jul 15 19:36:23.141189 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.140863 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a4ed2\"/\"sub-test-model-4a4ed2-kserve-self-signed-certs\"" Jul 15 19:36:23.150859 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.150838 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj"] Jul 15 19:36:23.276953 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.276920 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-dshm\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.277123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.276959 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr4m9\" (UniqueName: \"kubernetes.io/projected/a118b26e-63a5-41a2-a199-f6fa73bd99b8-kube-api-access-mr4m9\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.277123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.276981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tls-certs\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.277123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.277037 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tmp-dir\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.277123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.277065 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-model-cache\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.277123 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.277107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-home\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.378618 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.378573 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-dshm\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.378618 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.378622 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr4m9\" (UniqueName: \"kubernetes.io/projected/a118b26e-63a5-41a2-a199-f6fa73bd99b8-kube-api-access-mr4m9\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.378837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.378708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tls-certs\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.378837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.378762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tmp-dir\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.378837 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.378821 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-model-cache\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.378943 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.378879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-home\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.379187 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.379164 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tmp-dir\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.379286 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.379215 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-model-cache\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.379286 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.379275 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-home\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.380913 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.380889 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-dshm\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.381272 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.381255 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tls-certs\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.391125 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.391071 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr4m9\" (UniqueName: \"kubernetes.io/projected/a118b26e-63a5-41a2-a199-f6fa73bd99b8-kube-api-access-mr4m9\") pod \"sub-test-model-4a4ed2-kserve-868778fcc9-d8chj\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.447250 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.447207 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:23.604059 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:23.604034 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj"] Jul 15 19:36:23.605773 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:36:23.605747 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda118b26e_63a5_41a2_a199_f6fa73bd99b8.slice/crio-b4b2730e742ff8252bb2a87937fcef1bd5ef407dfcc80c380add9dc799c5cd15 WatchSource:0}: Error finding container b4b2730e742ff8252bb2a87937fcef1bd5ef407dfcc80c380add9dc799c5cd15: Status 404 returned error can't find the container with id b4b2730e742ff8252bb2a87937fcef1bd5ef407dfcc80c380add9dc799c5cd15 Jul 15 19:36:24.134250 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:24.134195 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" event={"ID":"a118b26e-63a5-41a2-a199-f6fa73bd99b8","Type":"ContainerStarted","Data":"000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9"} Jul 15 19:36:24.134695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:24.134260 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" event={"ID":"a118b26e-63a5-41a2-a199-f6fa73bd99b8","Type":"ContainerStarted","Data":"b4b2730e742ff8252bb2a87937fcef1bd5ef407dfcc80c380add9dc799c5cd15"} Jul 15 19:36:24.164906 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:24.164846 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" podStartSLOduration=1.16482979 podStartE2EDuration="1.16482979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:36:24.164047058 +0000 UTC m=+2153.631021370" watchObservedRunningTime="2026-07-15 19:36:24.16482979 +0000 UTC m=+2153.631804100" Jul 15 19:36:33.448219 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:33.448178 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:33.448700 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:33.448506 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:33.460787 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:33.460766 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:36:34.201333 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:36:34.201306 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:37:37.464988 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:37.464956 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj"] Jul 15 19:37:52.484774 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.484732 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" podUID="a118b26e-63a5-41a2-a199-f6fa73bd99b8" containerName="main" containerID="cri-o://000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9" gracePeriod=45 Jul 15 19:37:52.744734 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.744675 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:37:52.865989 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.865951 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tmp-dir\") pod \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " Jul 15 19:37:52.866172 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866001 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-model-cache\") pod \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " Jul 15 19:37:52.866172 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866030 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-home\") pod \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " Jul 15 19:37:52.866172 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866135 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-dshm\") pod \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " Jul 15 19:37:52.866172 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866166 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr4m9\" (UniqueName: \"kubernetes.io/projected/a118b26e-63a5-41a2-a199-f6fa73bd99b8-kube-api-access-mr4m9\") pod \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " Jul 15 19:37:52.866429 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866200 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tls-certs\") pod \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\" (UID: \"a118b26e-63a5-41a2-a199-f6fa73bd99b8\") " Jul 15 19:37:52.866429 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866289 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a118b26e-63a5-41a2-a199-f6fa73bd99b8" (UID: "a118b26e-63a5-41a2-a199-f6fa73bd99b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:52.866429 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866304 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a118b26e-63a5-41a2-a199-f6fa73bd99b8" (UID: "a118b26e-63a5-41a2-a199-f6fa73bd99b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:52.866429 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866394 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-home" (OuterVolumeSpecName: "home") pod "a118b26e-63a5-41a2-a199-f6fa73bd99b8" (UID: "a118b26e-63a5-41a2-a199-f6fa73bd99b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:52.866609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866560 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tmp-dir\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:37:52.866609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866579 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-model-cache\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:37:52.866609 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.866587 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-home\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:37:52.868459 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.868434 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a118b26e-63a5-41a2-a199-f6fa73bd99b8" (UID: "a118b26e-63a5-41a2-a199-f6fa73bd99b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:37:52.868599 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.868491 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-dshm" (OuterVolumeSpecName: "dshm") pod "a118b26e-63a5-41a2-a199-f6fa73bd99b8" (UID: "a118b26e-63a5-41a2-a199-f6fa73bd99b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:37:52.868599 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.868561 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a118b26e-63a5-41a2-a199-f6fa73bd99b8-kube-api-access-mr4m9" (OuterVolumeSpecName: "kube-api-access-mr4m9") pod "a118b26e-63a5-41a2-a199-f6fa73bd99b8" (UID: "a118b26e-63a5-41a2-a199-f6fa73bd99b8"). InnerVolumeSpecName "kube-api-access-mr4m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:37:52.967627 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.967587 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a118b26e-63a5-41a2-a199-f6fa73bd99b8-dshm\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:37:52.967627 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.967623 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr4m9\" (UniqueName: \"kubernetes.io/projected/a118b26e-63a5-41a2-a199-f6fa73bd99b8-kube-api-access-mr4m9\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:37:52.967627 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:52.967635 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a118b26e-63a5-41a2-a199-f6fa73bd99b8-tls-certs\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:37:53.573142 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.573107 2572 generic.go:356] "Generic (PLEG): container finished" podID="a118b26e-63a5-41a2-a199-f6fa73bd99b8" containerID="000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9" exitCode=0 Jul 15 19:37:53.573644 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.573154 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" Jul 15 19:37:53.573644 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.573196 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" event={"ID":"a118b26e-63a5-41a2-a199-f6fa73bd99b8","Type":"ContainerDied","Data":"000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9"} Jul 15 19:37:53.573644 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.573267 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj" event={"ID":"a118b26e-63a5-41a2-a199-f6fa73bd99b8","Type":"ContainerDied","Data":"b4b2730e742ff8252bb2a87937fcef1bd5ef407dfcc80c380add9dc799c5cd15"} Jul 15 19:37:53.573644 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.573285 2572 scope.go:117] "RemoveContainer" containerID="000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9" Jul 15 19:37:53.584115 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.584098 2572 scope.go:117] "RemoveContainer" containerID="000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9" Jul 15 19:37:53.584380 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:37:53.584360 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9\": container with ID starting with 000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9 not found: ID does not exist" containerID="000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9" Jul 15 19:37:53.584436 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.584393 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9"} err="failed to get container status \"000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9\": rpc error: code = NotFound desc = could not find container \"000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9\": container with ID starting with 000beab26a727afbd59e9d1865e50ab8a6d65c137f07eab1baf95b92ae3149f9 not found: ID does not exist" Jul 15 19:37:53.597737 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.597709 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj"] Jul 15 19:37:53.599274 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:53.599254 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4a4ed2/sub-test-model-4a4ed2-kserve-868778fcc9-d8chj"] Jul 15 19:37:55.034518 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:37:55.034484 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a118b26e-63a5-41a2-a199-f6fa73bd99b8" path="/var/lib/kubelet/pods/a118b26e-63a5-41a2-a199-f6fa73bd99b8/volumes" Jul 15 19:38:18.552247 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.552197 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm"] Jul 15 19:38:18.552842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.552735 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a118b26e-63a5-41a2-a199-f6fa73bd99b8" containerName="main" Jul 15 19:38:18.552842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.552755 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a118b26e-63a5-41a2-a199-f6fa73bd99b8" containerName="main" Jul 15 19:38:18.552842 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.552843 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a118b26e-63a5-41a2-a199-f6fa73bd99b8" containerName="main" Jul 15 19:38:18.555929 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.555904 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.560676 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.560655 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83827\"/\"default-dockercfg-j95w5\"" Jul 15 19:38:18.560676 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.560663 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83827\"/\"kube-root-ca.crt\"" Jul 15 19:38:18.560676 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.560674 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83827\"/\"openshift-service-ca.crt\"" Jul 15 19:38:18.560898 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.560665 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83827\"/\"rate-test-model-f83827-kserve-self-signed-certs\"" Jul 15 19:38:18.566042 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.566020 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm"] Jul 15 19:38:18.598075 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.598044 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tls-certs\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.598196 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.598089 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tmp-dir\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.598196 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.598129 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-dshm\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.598196 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.598156 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n2jn\" (UniqueName: \"kubernetes.io/projected/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-kube-api-access-6n2jn\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.598333 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.598229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-home\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.598333 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.598293 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-model-cache\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.699442 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-dshm\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.699695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n2jn\" (UniqueName: \"kubernetes.io/projected/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-kube-api-access-6n2jn\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.699695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699474 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-home\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.699695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699509 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-model-cache\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.699695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699573 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tls-certs\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.699695 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tmp-dir\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.700050 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699920 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-model-cache\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.700050 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.699967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-home\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.700050 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.700006 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tmp-dir\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.701753 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.701725 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-dshm\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.702160 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.702138 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tls-certs\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.708531 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.708508 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n2jn\" (UniqueName: \"kubernetes.io/projected/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-kube-api-access-6n2jn\") pod \"rate-test-model-f83827-kserve-7cf9c74b7f-t8frm\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:18.865854 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:18.865822 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:19.003495 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:19.003469 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm"] Jul 15 19:38:19.005126 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:38:19.005091 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29d3f81b_f3e9_47c8_b34d_a3624b76b1f6.slice/crio-d86700969362fb813f2ee158a25143c11843beb4ebdad0a4470f8ab0a72b2aa5 WatchSource:0}: Error finding container d86700969362fb813f2ee158a25143c11843beb4ebdad0a4470f8ab0a72b2aa5: Status 404 returned error can't find the container with id d86700969362fb813f2ee158a25143c11843beb4ebdad0a4470f8ab0a72b2aa5 Jul 15 19:38:19.698863 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:19.698825 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" event={"ID":"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6","Type":"ContainerStarted","Data":"8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014"} Jul 15 19:38:19.698863 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:19.698870 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" event={"ID":"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6","Type":"ContainerStarted","Data":"d86700969362fb813f2ee158a25143c11843beb4ebdad0a4470f8ab0a72b2aa5"} Jul 15 19:38:19.726556 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:19.726504 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" podStartSLOduration=1.726488861 podStartE2EDuration="1.726488861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:38:19.725952536 +0000 UTC m=+2269.192926847" watchObservedRunningTime="2026-07-15 19:38:19.726488861 +0000 UTC m=+2269.193463201" Jul 15 19:38:28.866735 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:28.866682 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:28.866735 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:28.866747 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:28.879204 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:28.879177 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:38:29.757380 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:38:29.757347 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:39:13.670813 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:13.670774 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm"] Jul 15 19:39:28.691869 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:28.691819 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" podUID="29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" containerName="main" containerID="cri-o://8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014" gracePeriod=45 Jul 15 19:39:28.945540 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:28.945470 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:39:29.027567 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.027524 2572 generic.go:356] "Generic (PLEG): container finished" podID="29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" containerID="8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014" exitCode=0 Jul 15 19:39:29.027764 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.027632 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" Jul 15 19:39:29.034221 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.034196 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" event={"ID":"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6","Type":"ContainerDied","Data":"8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014"} Jul 15 19:39:29.034373 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.034249 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm" event={"ID":"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6","Type":"ContainerDied","Data":"d86700969362fb813f2ee158a25143c11843beb4ebdad0a4470f8ab0a72b2aa5"} Jul 15 19:39:29.034373 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.034274 2572 scope.go:117] "RemoveContainer" containerID="8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014" Jul 15 19:39:29.036426 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036407 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-dshm\") pod \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " Jul 15 19:39:29.036511 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036442 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-model-cache\") pod \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " Jul 15 19:39:29.036511 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036495 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n2jn\" (UniqueName: \"kubernetes.io/projected/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-kube-api-access-6n2jn\") pod \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " Jul 15 19:39:29.036598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036511 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-home\") pod \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " Jul 15 19:39:29.036598 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036582 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tls-certs\") pod \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " Jul 15 19:39:29.036697 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036614 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tmp-dir\") pod \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\" (UID: \"29d3f81b-f3e9-47c8-b34d-a3624b76b1f6\") " Jul 15 19:39:29.036768 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036744 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" (UID: "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:39:29.036882 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036864 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-model-cache\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:29.036979 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036870 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-home" (OuterVolumeSpecName: "home") pod "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" (UID: "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:39:29.036979 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.036929 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" (UID: "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:39:29.038983 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.038916 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" (UID: "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:39:29.039103 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.039082 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-dshm" (OuterVolumeSpecName: "dshm") pod "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" (UID: "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:39:29.039167 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.039098 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-kube-api-access-6n2jn" (OuterVolumeSpecName: "kube-api-access-6n2jn") pod "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" (UID: "29d3f81b-f3e9-47c8-b34d-a3624b76b1f6"). InnerVolumeSpecName "kube-api-access-6n2jn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:39:29.043720 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.043681 2572 scope.go:117] "RemoveContainer" containerID="8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014" Jul 15 19:39:29.044063 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:39:29.044037 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014\": container with ID starting with 8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014 not found: ID does not exist" containerID="8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014" Jul 15 19:39:29.044148 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.044073 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014"} err="failed to get container status \"8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014\": rpc error: code = NotFound desc = could not find container \"8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014\": container with ID starting with 8b56762b167ad47b793e3ecc5a02b1b143b8821135e6d1cb6a16aa188554f014 not found: ID does not exist" Jul 15 19:39:29.137786 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.137745 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tls-certs\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:29.137786 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.137786 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-tmp-dir\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:29.137786 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.137798 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-dshm\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:29.138093 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.137810 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n2jn\" (UniqueName: \"kubernetes.io/projected/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-kube-api-access-6n2jn\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:29.138093 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.137824 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6-home\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:29.352275 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.352246 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm"] Jul 15 19:39:29.358648 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:29.358620 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f83827/rate-test-model-f83827-kserve-7cf9c74b7f-t8frm"] Jul 15 19:39:31.034109 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:31.034072 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" path="/var/lib/kubelet/pods/29d3f81b-f3e9-47c8-b34d-a3624b76b1f6/volumes" Jul 15 19:39:37.380708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.380673 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx"] Jul 15 19:39:37.381161 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.380899 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" podUID="47271a67-32ae-48a5-a2dd-9da2b6683b8d" containerName="maas-api" containerID="cri-o://d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9" gracePeriod=30 Jul 15 19:39:37.639574 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.639506 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:39:37.812768 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.812727 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47271a67-32ae-48a5-a2dd-9da2b6683b8d-openshift-service-ca\") pod \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " Jul 15 19:39:37.812939 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.812790 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls\") pod \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " Jul 15 19:39:37.812939 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.812808 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9hcw\" (UniqueName: \"kubernetes.io/projected/47271a67-32ae-48a5-a2dd-9da2b6683b8d-kube-api-access-t9hcw\") pod \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\" (UID: \"47271a67-32ae-48a5-a2dd-9da2b6683b8d\") " Jul 15 19:39:37.813165 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.813136 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47271a67-32ae-48a5-a2dd-9da2b6683b8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47271a67-32ae-48a5-a2dd-9da2b6683b8d" (UID: "47271a67-32ae-48a5-a2dd-9da2b6683b8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:39:37.814900 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.814875 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47271a67-32ae-48a5-a2dd-9da2b6683b8d" (UID: "47271a67-32ae-48a5-a2dd-9da2b6683b8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:39:37.814900 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.814894 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47271a67-32ae-48a5-a2dd-9da2b6683b8d-kube-api-access-t9hcw" (OuterVolumeSpecName: "kube-api-access-t9hcw") pod "47271a67-32ae-48a5-a2dd-9da2b6683b8d" (UID: "47271a67-32ae-48a5-a2dd-9da2b6683b8d"). InnerVolumeSpecName "kube-api-access-t9hcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:39:37.913655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.913577 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47271a67-32ae-48a5-a2dd-9da2b6683b8d-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:37.913655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.913608 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9hcw\" (UniqueName: \"kubernetes.io/projected/47271a67-32ae-48a5-a2dd-9da2b6683b8d-kube-api-access-t9hcw\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:37.913655 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:37.913619 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47271a67-32ae-48a5-a2dd-9da2b6683b8d-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:38.070339 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.070301 2572 generic.go:356] "Generic (PLEG): container finished" podID="47271a67-32ae-48a5-a2dd-9da2b6683b8d" containerID="d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9" exitCode=0 Jul 15 19:39:38.070513 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.070350 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" event={"ID":"47271a67-32ae-48a5-a2dd-9da2b6683b8d","Type":"ContainerDied","Data":"d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9"} Jul 15 19:39:38.070513 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.070374 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" Jul 15 19:39:38.070513 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.070412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx" event={"ID":"47271a67-32ae-48a5-a2dd-9da2b6683b8d","Type":"ContainerDied","Data":"437bfe9f149d139eb258a7d77b863ee42abcc88cf77b6dc8e74c0f7294b99b91"} Jul 15 19:39:38.070513 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.070433 2572 scope.go:117] "RemoveContainer" containerID="d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9" Jul 15 19:39:38.079084 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.079063 2572 scope.go:117] "RemoveContainer" containerID="d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9" Jul 15 19:39:38.079369 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:39:38.079349 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9\": container with ID starting with d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9 not found: ID does not exist" containerID="d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9" Jul 15 19:39:38.079445 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.079381 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9"} err="failed to get container status \"d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9\": rpc error: code = NotFound desc = could not find container \"d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9\": container with ID starting with d13308147bab666f0dd1b9589870341022f9eb544597f996619454a353aff7f9 not found: ID does not exist" Jul 15 19:39:38.095146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.095111 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx"] Jul 15 19:39:38.099478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:38.099458 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-4a4ed2-869f9d6844-fv4cx"] Jul 15 19:39:39.034244 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:39.034204 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47271a67-32ae-48a5-a2dd-9da2b6683b8d" path="/var/lib/kubelet/pods/47271a67-32ae-48a5-a2dd-9da2b6683b8d/volumes" Jul 15 19:39:44.997743 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:44.997707 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9"] Jul 15 19:39:44.998262 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:44.998061 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" podUID="ba022e91-d3ac-4241-90bd-494af8e3fec0" containerName="maas-api" containerID="cri-o://bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b" gracePeriod=30 Jul 15 19:39:45.450781 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.450756 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:39:45.583090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.583006 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cth2h\" (UniqueName: \"kubernetes.io/projected/ba022e91-d3ac-4241-90bd-494af8e3fec0-kube-api-access-cth2h\") pod \"ba022e91-d3ac-4241-90bd-494af8e3fec0\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " Jul 15 19:39:45.583090 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.583051 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba022e91-d3ac-4241-90bd-494af8e3fec0-maas-api-tls\") pod \"ba022e91-d3ac-4241-90bd-494af8e3fec0\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " Jul 15 19:39:45.583381 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.583106 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba022e91-d3ac-4241-90bd-494af8e3fec0-openshift-service-ca\") pod \"ba022e91-d3ac-4241-90bd-494af8e3fec0\" (UID: \"ba022e91-d3ac-4241-90bd-494af8e3fec0\") " Jul 15 19:39:45.583562 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.583536 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba022e91-d3ac-4241-90bd-494af8e3fec0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba022e91-d3ac-4241-90bd-494af8e3fec0" (UID: "ba022e91-d3ac-4241-90bd-494af8e3fec0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:39:45.585122 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.585093 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba022e91-d3ac-4241-90bd-494af8e3fec0-kube-api-access-cth2h" (OuterVolumeSpecName: "kube-api-access-cth2h") pod "ba022e91-d3ac-4241-90bd-494af8e3fec0" (UID: "ba022e91-d3ac-4241-90bd-494af8e3fec0"). InnerVolumeSpecName "kube-api-access-cth2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:39:45.585358 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.585186 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba022e91-d3ac-4241-90bd-494af8e3fec0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba022e91-d3ac-4241-90bd-494af8e3fec0" (UID: "ba022e91-d3ac-4241-90bd-494af8e3fec0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:39:45.684219 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.684180 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cth2h\" (UniqueName: \"kubernetes.io/projected/ba022e91-d3ac-4241-90bd-494af8e3fec0-kube-api-access-cth2h\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:45.684219 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.684216 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba022e91-d3ac-4241-90bd-494af8e3fec0-maas-api-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:45.684219 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:45.684225 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba022e91-d3ac-4241-90bd-494af8e3fec0-openshift-service-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 15 19:39:46.110146 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.110106 2572 generic.go:356] "Generic (PLEG): container finished" podID="ba022e91-d3ac-4241-90bd-494af8e3fec0" containerID="bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b" exitCode=0 Jul 15 19:39:46.110656 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.110156 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" Jul 15 19:39:46.110656 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.110171 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" event={"ID":"ba022e91-d3ac-4241-90bd-494af8e3fec0","Type":"ContainerDied","Data":"bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b"} Jul 15 19:39:46.110656 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.110218 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9" event={"ID":"ba022e91-d3ac-4241-90bd-494af8e3fec0","Type":"ContainerDied","Data":"1113377e2a0f1f56906dc943d5682f5f606e10349f2511ba2be08043610c9f37"} Jul 15 19:39:46.110656 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.110266 2572 scope.go:117] "RemoveContainer" containerID="bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b" Jul 15 19:39:46.119850 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.119815 2572 scope.go:117] "RemoveContainer" containerID="bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b" Jul 15 19:39:46.120094 ip-10-0-137-168 kubenswrapper[2572]: E0715 19:39:46.120071 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b\": container with ID starting with bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b not found: ID does not exist" containerID="bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b" Jul 15 19:39:46.120150 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.120101 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b"} err="failed to get container status \"bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b\": rpc error: code = NotFound desc = could not find container \"bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b\": container with ID starting with bd6517f4575c3473c011c60c0737ecbacbcd11752eb8ea1084b4910320494d0b not found: ID does not exist" Jul 15 19:39:46.134315 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.134287 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9"] Jul 15 19:39:46.137213 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:46.137159 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f83827-7664c9d979-r9kj9"] Jul 15 19:39:47.039014 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:47.038977 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba022e91-d3ac-4241-90bd-494af8e3fec0" path="/var/lib/kubelet/pods/ba022e91-d3ac-4241-90bd-494af8e3fec0/volumes" Jul 15 19:39:58.227744 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:58.227705 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c89bff45-p4cdd_736ad68d-b406-44fe-90b5-d74c095c812e/copy-manifests/0.log" Jul 15 19:39:58.240862 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:58.240827 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c89bff45-p4cdd_736ad68d-b406-44fe-90b5-d74c095c812e/manager/0.log" Jul 15 19:39:58.831611 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:58.831582 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735715-6v94z_16bf84bb-016a-4c36-804d-b775f456154a/cleanup/0.log" Jul 15 19:39:58.939628 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:39:58.939598 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735730-zcfn2_7b969d42-e293-4950-a9bc-88c2838a39ee/cleanup/0.log" Jul 15 19:40:00.921075 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:00.921028 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c7whk_fa9c7475-0712-4567-8833-2168614b3547/manager/0.log" Jul 15 19:40:01.646382 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:01.646352 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8c5qn_bfd3bd0a-8538-411c-b895-a53e947b90e4/discovery/0.log" Jul 15 19:40:02.248229 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:02.248186 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-thpfd_8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48/payload-processing/0.log" Jul 15 19:40:02.359304 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:02.359261 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dffcc884d-qt2hp_b656aa6d-2d76-41d4-a398-cc3bfc725624/router/0.log" Jul 15 19:40:02.807132 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:02.807097 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gffgr_0c8980fe-caa3-4142-8d74-2a3db5d73656/main/0.log" Jul 15 19:40:11.875034 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:11.875005 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b4dg5_35b4663c-ea09-4be7-a31c-660e76c46799/global-pull-secret-syncer/0.log" Jul 15 19:40:12.008760 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:12.008722 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8xkv8_cfbd346b-c505-44fb-828c-0fc38124c540/konnectivity-agent/0.log" Jul 15 19:40:12.083441 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:12.083390 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-168.ec2.internal_631442f90a9aec34b0d279ea0f5f9c73/haproxy/0.log" Jul 15 19:40:17.174989 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:17.174957 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c7whk_fa9c7475-0712-4567-8833-2168614b3547/manager/0.log" Jul 15 19:40:19.235698 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:19.235613 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-wgxbr_613706c9-971d-4afb-bec1-093b211a3b51/cluster-monitoring-operator/0.log" Jul 15 19:40:19.510248 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:19.510149 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkxdv_e8191ae3-b823-45c7-a04a-129e72a1b0d5/node-exporter/0.log" Jul 15 19:40:19.534916 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:19.534880 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkxdv_e8191ae3-b823-45c7-a04a-129e72a1b0d5/kube-rbac-proxy/0.log" Jul 15 19:40:19.563053 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:19.563022 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkxdv_e8191ae3-b823-45c7-a04a-129e72a1b0d5/init-textfile/0.log" Jul 15 19:40:20.029289 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:20.029256 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-9pnfs_40b05f0f-c537-49f0-99f8-c51506a2a09a/prometheus-operator-admission-webhook/0.log" Jul 15 19:40:21.482213 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482181 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv"] Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482516 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba022e91-d3ac-4241-90bd-494af8e3fec0" containerName="maas-api" Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482528 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba022e91-d3ac-4241-90bd-494af8e3fec0" containerName="maas-api" Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482540 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47271a67-32ae-48a5-a2dd-9da2b6683b8d" containerName="maas-api" Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482546 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="47271a67-32ae-48a5-a2dd-9da2b6683b8d" containerName="maas-api" Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482561 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" containerName="main" Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482566 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" containerName="main" Jul 15 19:40:21.482615 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482617 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="29d3f81b-f3e9-47c8-b34d-a3624b76b1f6" containerName="main" Jul 15 19:40:21.482845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482628 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="47271a67-32ae-48a5-a2dd-9da2b6683b8d" containerName="maas-api" Jul 15 19:40:21.482845 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.482635 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba022e91-d3ac-4241-90bd-494af8e3fec0" containerName="maas-api" Jul 15 19:40:21.485364 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.485333 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.488099 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.488076 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-csbwp\"/\"default-dockercfg-cx64j\"" Jul 15 19:40:21.488630 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.488611 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-csbwp\"/\"openshift-service-ca.crt\"" Jul 15 19:40:21.489166 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.489143 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-csbwp\"/\"kube-root-ca.crt\"" Jul 15 19:40:21.495764 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.495741 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv"] Jul 15 19:40:21.592136 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.592103 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-lib-modules\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.592334 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.592152 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-sys\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.592334 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.592226 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-proc\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.592334 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.592294 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-podres\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.592488 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.592347 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnp67\" (UniqueName: \"kubernetes.io/projected/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-kube-api-access-wnp67\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693007 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.692976 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-sys\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693024 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-proc\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693063 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-podres\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693097 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnp67\" (UniqueName: \"kubernetes.io/projected/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-kube-api-access-wnp67\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693138 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-proc\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693175 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-lib-modules\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693385 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693099 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-sys\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693385 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693249 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-podres\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.693385 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.693266 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-lib-modules\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.701977 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.701954 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnp67\" (UniqueName: \"kubernetes.io/projected/d4a9d160-0cc5-40ca-a54b-9829d2a26ffa-kube-api-access-wnp67\") pod \"perf-node-gather-daemonset-4z7mv\" (UID: \"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa\") " pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.795742 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.795653 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:21.929268 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:21.929225 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv"] Jul 15 19:40:21.931506 ip-10-0-137-168 kubenswrapper[2572]: W0715 19:40:21.931477 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd4a9d160_0cc5_40ca_a54b_9829d2a26ffa.slice/crio-0df9933e551f3cab5fb5c208e0d3819cba3b4b0880615c8028f3bb484f043bdb WatchSource:0}: Error finding container 0df9933e551f3cab5fb5c208e0d3819cba3b4b0880615c8028f3bb484f043bdb: Status 404 returned error can't find the container with id 0df9933e551f3cab5fb5c208e0d3819cba3b4b0880615c8028f3bb484f043bdb Jul 15 19:40:22.151589 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.151555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:40:22.155548 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.155530 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/3.log" Jul 15 19:40:22.281568 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.281537 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" event={"ID":"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa","Type":"ContainerStarted","Data":"25e6aad4496e1635255bf9874b76102bc581142dc6b681e00813c18d982e696e"} Jul 15 19:40:22.281727 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.281590 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" event={"ID":"d4a9d160-0cc5-40ca-a54b-9829d2a26ffa","Type":"ContainerStarted","Data":"0df9933e551f3cab5fb5c208e0d3819cba3b4b0880615c8028f3bb484f043bdb"} Jul 15 19:40:22.281727 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.281624 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:22.298085 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.298041 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" podStartSLOduration=1.298026198 podStartE2EDuration="1.298026198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:40:22.297453531 +0000 UTC m=+2391.764427844" watchObservedRunningTime="2026-07-15 19:40:22.298026198 +0000 UTC m=+2391.765000509" Jul 15 19:40:22.728874 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:22.728834 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-tvm92_de4044c8-ecfb-4a1f-b543-e2bca312be46/download-server/0.log" Jul 15 19:40:23.334846 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:23.334818 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-tgvwf_ec80b08a-052a-4f8c-9536-fcb0121936cb/volume-data-source-validator/0.log" Jul 15 19:40:24.501014 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:24.500988 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z65bn_9de406e7-cfc2-4a52-b55b-c7819d750e2a/dns/0.log" Jul 15 19:40:24.526891 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:24.526865 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z65bn_9de406e7-cfc2-4a52-b55b-c7819d750e2a/kube-rbac-proxy/0.log" Jul 15 19:40:24.554520 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:24.554494 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2npnt_6196293c-156c-4f46-9f14-f228ec4b2faa/dns-node-resolver/0.log" Jul 15 19:40:25.211010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:25.210976 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dz2bk_c6d5c01e-3cbb-476f-b39c-9960a8e4beb7/node-ca/0.log" Jul 15 19:40:26.342228 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:26.342193 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8c5qn_bfd3bd0a-8538-411c-b895-a53e947b90e4/discovery/0.log" Jul 15 19:40:26.567412 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:26.567372 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-thpfd_8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48/payload-processing/0.log" Jul 15 19:40:26.591026 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:26.590996 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dffcc884d-qt2hp_b656aa6d-2d76-41d4-a398-cc3bfc725624/router/0.log" Jul 15 19:40:27.303157 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:27.303125 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c79mn_dba7f537-3e9b-41bc-ba46-cf83477ec8a4/serve-healthcheck-canary/0.log" Jul 15 19:40:27.824003 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:27.823971 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hlmwm_3467ed9c-093c-4b71-b519-3dffa22383eb/insights-operator/0.log" Jul 15 19:40:27.824489 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:27.824475 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hlmwm_3467ed9c-093c-4b71-b519-3dffa22383eb/insights-operator/1.log" Jul 15 19:40:28.048935 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:28.048907 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xlj2j_183b0d00-adc2-4d7a-806b-916a026874f1/kube-rbac-proxy/0.log" Jul 15 19:40:28.093805 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:28.093730 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xlj2j_183b0d00-adc2-4d7a-806b-916a026874f1/exporter/0.log" Jul 15 19:40:28.140363 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:28.140335 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xlj2j_183b0d00-adc2-4d7a-806b-916a026874f1/extractor/0.log" Jul 15 19:40:28.296650 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:28.296622 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-csbwp/perf-node-gather-daemonset-4z7mv" Jul 15 19:40:31.180506 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:31.180481 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:40:31.187849 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:31.187828 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:40:31.190507 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:31.190490 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-2kz84_cf3ec3be-080b-4691-838d-2a72b6e7192e/console-operator/2.log" Jul 15 19:40:31.196790 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:31.196772 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fn4z6_7bdd365b-6d8b-4b34-b68f-0d17f9600606/ovn-acl-logging/0.log" Jul 15 19:40:31.542684 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:31.542653 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c89bff45-p4cdd_736ad68d-b406-44fe-90b5-d74c095c812e/manager/0.log" Jul 15 19:40:31.569697 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:31.569666 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c89bff45-p4cdd_736ad68d-b406-44fe-90b5-d74c095c812e/copy-manifests/0.log" Jul 15 19:40:32.727560 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:32.727530 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8c5qn_bfd3bd0a-8538-411c-b895-a53e947b90e4/discovery/0.log" Jul 15 19:40:32.904187 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:32.904110 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-thpfd_8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48/payload-processing/0.log" Jul 15 19:40:32.924381 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:32.924353 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dffcc884d-qt2hp_b656aa6d-2d76-41d4-a398-cc3bfc725624/router/0.log" Jul 15 19:40:32.933017 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:32.932995 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8659b4c98-fdkvl_6e59a746-0cf4-4939-8448-7ee97b78fbbf/manager/0.log" Jul 15 19:40:34.262508 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:34.262468 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8c5qn_bfd3bd0a-8538-411c-b895-a53e947b90e4/discovery/0.log" Jul 15 19:40:34.449063 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:34.449035 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-thpfd_8a8e8a5d-50ae-4148-aca3-afbc6a4b5d48/payload-processing/0.log" Jul 15 19:40:34.463478 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:34.463453 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dffcc884d-qt2hp_b656aa6d-2d76-41d4-a398-cc3bfc725624/router/0.log" Jul 15 19:40:37.983865 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:37.983780 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-hh4jh_1a17900a-1ce3-4955-8601-9cf3efe6b24d/migrator/0.log" Jul 15 19:40:38.010708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:38.010673 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-hh4jh_1a17900a-1ce3-4955-8601-9cf3efe6b24d/graceful-termination/0.log" Jul 15 19:40:38.448010 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:38.447971 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-fjxtl_176a3fce-370b-48ee-b6b2-71302f9d3731/kube-storage-version-migrator-operator/1.log" Jul 15 19:40:38.448708 ip-10-0-137-168 kubenswrapper[2572]: I0715 19:40:38.448687 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-fjxtl_176a3fce-370b-48ee-b6b2-71302f9d3731/kube-storage-version-migrator-operator/0.log"