Jul 20 13:01:29.334153 ip-10-0-131-198 systemd[1]: Starting Kubernetes Kubelet... Jul 20 13:01:29.721014 ip-10-0-131-198 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 13:01:29.721014 ip-10-0-131-198 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 13:01:29.721014 ip-10-0-131-198 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 13:01:29.721014 ip-10-0-131-198 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 13:01:29.721014 ip-10-0-131-198 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 13:01:29.722247 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.722118 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 13:01:29.727068 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727053 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 13:01:29.727068 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727068 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 13:01:29.727068 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727071 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727074 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727077 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727086 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727089 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727092 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727094 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727097 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727099 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727103 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727106 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727108 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727111 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727113 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727116 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727118 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727121 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727123 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727126 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727128 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 13:01:29.727170 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727131 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727134 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727138 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727142 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727145 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727148 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727151 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727153 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727156 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727159 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727162 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727165 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727168 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727170 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727174 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727177 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727180 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727183 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727186 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 13:01:29.727638 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727188 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727191 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727193 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727196 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727198 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727201 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727204 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727206 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727209 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727211 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727214 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727217 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727220 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727223 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727225 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727228 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727230 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727233 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727235 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727238 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 13:01:29.728105 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727240 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727243 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727245 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727248 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727250 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727254 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727257 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727260 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727263 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727266 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727269 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727272 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727274 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727277 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727280 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727284 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727287 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727290 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727293 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 13:01:29.728617 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727296 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727299 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727302 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727304 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727307 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727311 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727315 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727318 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727321 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727324 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727326 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727329 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727332 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727334 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727337 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727339 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727342 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727345 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.727347 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 13:01:29.729099 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728053 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728060 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728064 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728066 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728069 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728072 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728075 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728078 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728081 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728084 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728087 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728089 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728092 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728094 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728097 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728099 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728102 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728105 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728107 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728111 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 13:01:29.729568 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728114 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728116 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728119 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728122 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728125 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728127 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728130 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728132 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728135 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728137 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728140 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728142 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728145 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728148 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728150 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728155 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728158 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728161 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728165 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 13:01:29.730070 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728169 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728172 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728174 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728177 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728181 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728183 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728186 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728188 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728191 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728193 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728196 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728199 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728201 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728204 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728207 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728209 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728212 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728214 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728217 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728219 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 13:01:29.730540 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728222 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728224 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728227 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728229 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728232 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728235 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728238 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728241 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728244 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728246 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728249 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728251 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728254 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728257 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728260 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728262 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728265 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728268 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728272 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 13:01:29.731041 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728275 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728279 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728281 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728284 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728287 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728291 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728293 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728296 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728299 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728302 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728304 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728307 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728310 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728312 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728315 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728317 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728320 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728323 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728325 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728328 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 13:01:29.731511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728330 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728686 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728691 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728694 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728696 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728699 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728702 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728705 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728708 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728710 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728713 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728716 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728718 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728721 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728723 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728726 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728729 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728734 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728737 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728739 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 13:01:29.732012 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728742 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728745 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728747 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728750 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728753 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728755 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728758 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728760 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728763 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728765 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728768 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728770 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728773 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728775 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728778 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728782 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728786 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728789 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728792 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728795 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 13:01:29.732505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728798 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728800 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728803 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728805 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728808 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728810 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728813 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728816 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728819 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728823 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728826 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728828 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728831 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728833 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728837 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728839 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728842 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728844 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728847 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 13:01:29.732995 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728849 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728852 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728855 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728859 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728861 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728864 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728867 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728869 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728872 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728888 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728891 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728893 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728897 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728900 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728902 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728905 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728909 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728911 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728914 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 13:01:29.733480 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728918 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728922 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728925 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728929 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728933 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728935 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728938 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728941 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728944 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728947 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728949 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728952 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728954 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728957 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728960 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728962 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728965 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728968 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728970 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 13:01:29.733985 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728973 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728975 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.728978 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729046 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729057 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729063 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729068 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729072 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729076 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729080 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729084 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729087 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729090 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729095 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729098 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729102 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729107 2564 flags.go:64] FLAG: --cgroup-root="" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729110 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729113 2564 flags.go:64] FLAG: --client-ca-file="" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729116 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729119 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729123 2564 flags.go:64] FLAG: --cluster-domain="" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729126 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 13:01:29.734441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729129 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729133 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729136 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729140 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729143 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729146 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729149 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729152 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729155 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729158 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729161 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729164 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729168 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729170 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729173 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729176 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729179 2564 flags.go:64] FLAG: --enable-server="true" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729183 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729187 2564 flags.go:64] FLAG: --event-burst="100" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729190 2564 flags.go:64] FLAG: --event-qps="50" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729193 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729196 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729199 2564 flags.go:64] FLAG: --eviction-hard="" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729203 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 13:01:29.735077 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729206 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729209 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729212 2564 flags.go:64] FLAG: --eviction-soft="" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729215 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729218 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729221 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729224 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729227 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729230 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729233 2564 flags.go:64] FLAG: --feature-gates="" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729237 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729240 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729243 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729246 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729249 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729252 2564 flags.go:64] FLAG: --help="false" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729255 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-131-198.ec2.internal" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729258 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729261 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729264 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729267 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729270 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729273 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 13:01:29.735722 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729276 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729278 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729281 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729284 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729287 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729290 2564 flags.go:64] FLAG: --kube-reserved="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729293 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729296 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729300 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729303 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729305 2564 flags.go:64] FLAG: --lock-file="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729308 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729311 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729314 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729319 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729322 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729325 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729328 2564 flags.go:64] FLAG: --logging-format="text" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729331 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729334 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729337 2564 flags.go:64] FLAG: --manifest-url="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729340 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729344 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729347 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729352 2564 flags.go:64] FLAG: --max-pods="110" Jul 20 13:01:29.736320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729355 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729358 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729361 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729364 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729367 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729370 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729373 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729380 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729383 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729386 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729389 2564 flags.go:64] FLAG: --pod-cidr="" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729393 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729398 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729401 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729404 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729407 2564 flags.go:64] FLAG: --port="10250" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729411 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729414 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-044f50ff2f44bde32" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729417 2564 flags.go:64] FLAG: --qos-reserved="" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729420 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729423 2564 flags.go:64] FLAG: --register-node="true" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729425 2564 flags.go:64] FLAG: --register-with-taints="" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729429 2564 flags.go:64] FLAG: --registry-burst="10" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729432 2564 flags.go:64] FLAG: --registry-qps="5" Jul 20 13:01:29.736981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729434 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729437 2564 flags.go:64] FLAG: --reserved-memory="" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729440 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729444 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729447 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729450 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729452 2564 flags.go:64] FLAG: --runonce="false" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729455 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729458 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729461 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729464 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729467 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729470 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729473 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729476 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729479 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729482 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729485 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729487 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729498 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729503 2564 flags.go:64] FLAG: --system-cgroups="" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729506 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729511 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729514 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729517 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 13:01:29.737578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729522 2564 flags.go:64] FLAG: --tls-min-version="" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729525 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729527 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729530 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729533 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729536 2564 flags.go:64] FLAG: --v="2" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729539 2564 flags.go:64] FLAG: --version="false" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729543 2564 flags.go:64] FLAG: --vmodule="" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729547 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.729551 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729660 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729663 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729666 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729669 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729673 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729677 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729680 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729683 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729686 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729689 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729692 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 13:01:29.738396 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729695 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729699 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729702 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729705 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729708 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729711 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729714 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729717 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729721 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729724 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729728 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729730 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729733 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729735 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729738 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729741 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729743 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729746 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729748 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729751 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 13:01:29.739060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729754 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729756 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729759 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729762 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729764 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729767 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729769 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729772 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729774 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729777 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729781 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729784 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729787 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729790 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729792 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729795 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729798 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729800 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729802 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 13:01:29.739548 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729805 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729810 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729813 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729818 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729821 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729824 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729827 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729829 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729832 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729834 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729837 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729839 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729842 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729844 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729847 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729850 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729852 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729855 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729858 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729860 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 13:01:29.740027 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729863 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729865 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729868 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729870 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729887 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729891 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729893 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729896 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729898 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729901 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729904 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729906 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729909 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729913 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729916 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729920 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729923 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729925 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729928 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729930 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 13:01:29.740505 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729933 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729936 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729938 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729941 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729944 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729947 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729950 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729952 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.729955 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.730510 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.737003 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.737018 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.737037 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.737043 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 13:01:29.741000 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737098 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737102 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737106 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737110 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737115 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737118 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737121 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737124 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737127 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737130 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737133 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737135 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737138 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737141 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737143 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737146 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737149 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737151 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737154 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 13:01:29.741349 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737157 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737159 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737161 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737165 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737169 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737172 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737175 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737177 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737180 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737182 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737187 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737190 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737192 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737195 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737197 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737200 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737202 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737205 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737207 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737210 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 13:01:29.741823 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737212 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737215 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737217 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737220 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737223 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737225 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737228 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737230 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737233 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737235 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737238 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737240 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737244 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737246 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737249 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737251 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737254 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737256 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737259 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737261 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 13:01:29.742328 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737264 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737266 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737270 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737274 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737276 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737278 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737281 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737285 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737288 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737292 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737294 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737297 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737300 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737303 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737305 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737308 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737310 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737313 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737316 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 13:01:29.742855 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737318 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737321 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737323 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737326 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737328 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737331 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737333 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737336 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737338 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737341 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737343 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737345 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737348 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737350 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737353 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737357 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737360 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737363 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737365 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737368 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 13:01:29.743344 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737371 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.737376 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737487 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737502 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737506 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737509 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737512 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737515 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737518 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737522 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737526 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737528 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737531 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737534 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 13:01:29.743822 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737537 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737540 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737542 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737545 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737548 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737552 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737555 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737558 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737561 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737564 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737567 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737570 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737572 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737575 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737579 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737581 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737585 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737588 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737591 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 13:01:29.744193 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737593 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737596 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737599 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737601 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737604 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737606 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737609 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737612 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737614 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737617 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737619 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737622 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737624 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737627 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737630 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737632 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737635 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737638 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737640 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737643 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 13:01:29.744636 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737645 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737648 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737650 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737653 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737656 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737658 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737661 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737663 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737666 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737669 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737672 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737674 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737677 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737679 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737682 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737684 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737687 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737689 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737692 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 13:01:29.745146 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737694 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737697 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737701 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737704 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737706 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737709 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737711 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737714 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737716 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737719 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737721 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737724 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737726 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737729 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737731 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737734 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737736 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737739 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737742 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737744 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 13:01:29.745613 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737747 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737750 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737752 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737755 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737758 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737760 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737763 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737765 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:29.737767 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.737772 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.740362 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.742374 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.743904 2564 server.go:1019] "Starting client certificate rotation" Jul 20 13:01:29.746144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.744001 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 13:01:29.746491 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.744035 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 13:01:29.746491 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.746374 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 13:01:29.747507 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.747488 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 13:01:29.756663 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.756645 2564 log.go:25] "Validated CRI v1 runtime API" Jul 20 13:01:29.760041 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.760024 2564 log.go:25] "Validated CRI v1 image API" Jul 20 13:01:29.761649 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.761634 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 13:01:29.766478 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.766456 2564 fs.go:135] Filesystem UUIDs: map[7418b4d5-0d0b-4dd4-a949-b4c24ac002ee:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 afd0b679-ad4d-4c24-9001-aeb3999bf6e2:/dev/nvme0n1p4] Jul 20 13:01:29.766529 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.766478 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 13:01:29.772291 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.772165 2564 manager.go:216] Machine: {Timestamp:2026-07-20 13:01:29.770086176 +0000 UTC m=+0.321978887 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095887 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ef3d91111b463c77cae40f780a22e SystemUUID:ec2ef3d9-1111-b463-c77c-ae40f780a22e BootID:9b4e3243-6bb4-469e-b29f-d36f3e76e60f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:46:31:72:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:46:31:72:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:bc:79:6d:d1:ab Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 13:01:29.772291 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.772281 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 13:01:29.772459 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.772404 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 13:01:29.772513 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.772495 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 13:01:29.775373 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.775348 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 13:01:29.775541 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.775377 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-198.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 13:01:29.776550 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.776537 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 13:01:29.776606 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.776557 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 13:01:29.776606 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.776567 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 13:01:29.777201 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.777189 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 13:01:29.778754 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.778742 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 13:01:29.781375 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781362 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 20 13:01:29.781513 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781502 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 13:01:29.781582 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781540 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 20 13:01:29.781582 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781556 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 13:01:29.781582 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781576 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 13:01:29.781700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781590 2564 kubelet.go:398] "Adding apiserver pod source" Jul 20 13:01:29.781700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.781605 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 13:01:29.782039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.782026 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 13:01:29.782118 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.782060 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 13:01:29.784347 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.784261 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 13:01:29.786395 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.786375 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 13:01:29.786481 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.786416 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 13:01:29.787443 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787429 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787447 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787456 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787464 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787476 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787484 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787495 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787503 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787513 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 13:01:29.787519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787521 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 13:01:29.787905 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787534 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 13:01:29.787905 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787543 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 13:01:29.787905 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787574 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 13:01:29.787905 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.787583 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 13:01:29.790817 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.790801 2564 server.go:1268] "Started kubelet" Jul 20 13:01:29.790940 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.790912 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 13:01:29.790999 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.790913 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 13:01:29.790999 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.790986 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 13:01:29.791473 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.791406 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-198.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 13:01:29.791473 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.791411 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 13:01:29.791473 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.791435 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 13:01:29.791717 ip-10-0-131-198 systemd[1]: Started Kubernetes Kubelet. Jul 20 13:01:29.792819 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.792687 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 13:01:29.793118 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.793103 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 20 13:01:29.795996 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.795977 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 13:01:29.796520 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.795616 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-198.ec2.internal.18c40108d504a1e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-198.ec2.internal,UID:ip-10-0-131-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-198.ec2.internal,},FirstTimestamp:2026-07-20 13:01:29.790775783 +0000 UTC m=+0.342668472,LastTimestamp:2026-07-20 13:01:29.790775783 +0000 UTC m=+0.342668472,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-198.ec2.internal,}" Jul 20 13:01:29.798298 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.798277 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 13:01:29.798298 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.798288 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 13:01:29.798456 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.798383 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:29.798899 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.798859 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 13:01:29.798899 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.798860 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 13:01:29.798899 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.798902 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 13:01:29.799074 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799025 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 20 13:01:29.799074 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799033 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 20 13:01:29.799166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799095 2564 factory.go:55] Registering systemd factory Jul 20 13:01:29.799166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799111 2564 factory.go:223] Registration of the systemd container factory successfully Jul 20 13:01:29.799320 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799306 2564 factory.go:153] Registering CRI-O factory Jul 20 13:01:29.799379 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799324 2564 factory.go:223] Registration of the crio container factory successfully Jul 20 13:01:29.799428 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799388 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 13:01:29.799428 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799410 2564 factory.go:103] Registering Raw factory Jul 20 13:01:29.799428 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799424 2564 manager.go:1196] Started watching for new ooms in manager Jul 20 13:01:29.799838 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.799825 2564 manager.go:349] Starting recovery of all containers Jul 20 13:01:29.809205 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.809099 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-198.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 13:01:29.809205 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.809180 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 13:01:29.810842 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.810826 2564 manager.go:354] Recovery completed Jul 20 13:01:29.813064 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.813045 2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 20 13:01:29.816042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.816030 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:29.817886 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.817850 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:29.817946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.817927 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:29.817946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.817939 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:29.818456 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.818442 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 13:01:29.818456 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.818454 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:29.818542 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.818455 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 13:01:29.818542 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.818521 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 20 13:01:29.819127 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.819112 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:29.819186 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.819137 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:29.819186 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.819149 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:29.819740 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.819728 2564 policy_none.go:49] "None policy: Start" Jul 20 13:01:29.819785 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.819745 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 13:01:29.819785 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.819755 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 13:01:29.819912 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.819788 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-198.ec2.internal.18c40108d6a212d8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-198.ec2.internal,UID:ip-10-0-131-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-198.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-198.ec2.internal,},FirstTimestamp:2026-07-20 13:01:29.817871064 +0000 UTC m=+0.369763757,LastTimestamp:2026-07-20 13:01:29.817871064 +0000 UTC m=+0.369763757,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-198.ec2.internal,}" Jul 20 13:01:29.820773 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.820763 2564 policy_none.go:47] "Start" Jul 20 13:01:29.820986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.820967 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmkf2" Jul 20 13:01:29.827959 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.827773 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-198.ec2.internal.18c40108d6a3058b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-198.ec2.internal,UID:ip-10-0-131-198.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-198.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-198.ec2.internal,},FirstTimestamp:2026-07-20 13:01:29.817933195 +0000 UTC m=+0.369825885,LastTimestamp:2026-07-20 13:01:29.817933195 +0000 UTC m=+0.369825885,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-198.ec2.internal,}" Jul 20 13:01:29.828804 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.828789 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmkf2" Jul 20 13:01:29.865449 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865435 2564 manager.go:337] "Starting Device Plugin manager" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.865464 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865475 2564 server.go:87] "Starting device plugin registration server" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865699 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865713 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865861 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865954 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.865962 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.866426 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 13:01:29.870169 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.866475 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:29.928137 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.928112 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 13:01:29.929297 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.929277 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 13:01:29.929373 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.929310 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 13:01:29.929373 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.929329 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 13:01:29.929373 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.929361 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 13:01:29.931742 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.931723 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 13:01:29.966674 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.966658 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:29.967459 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.967445 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:29.967545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.967471 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:29.967545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.967481 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:29.967545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.967502 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-198.ec2.internal" Jul 20 13:01:29.976637 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:29.976590 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-198.ec2.internal" Jul 20 13:01:29.976637 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.976610 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-198.ec2.internal\": node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:29.999480 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:29.999455 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.029773 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.029753 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal"] Jul 20 13:01:30.029833 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.029807 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:30.030606 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.030590 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:30.030691 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.030626 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:30.030691 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.030641 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:30.032152 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032138 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:30.032258 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.032302 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032273 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:30.032860 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032843 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:30.032929 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032870 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:30.032929 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032892 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:30.032929 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032904 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:30.032929 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032908 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:30.032929 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.032917 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:30.034612 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.034596 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.034698 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.034621 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:30.035550 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.035535 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:30.035638 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.035559 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:30.035638 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.035572 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:30.058577 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.058556 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-198.ec2.internal\" not found" node="ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.062976 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.062960 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-198.ec2.internal\" not found" node="ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.100154 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.100124 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.100241 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.100178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df743b01d5059c4d74a994cdac691aa0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal\" (UID: \"df743b01d5059c4d74a994cdac691aa0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.100241 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.100203 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df743b01d5059c4d74a994cdac691aa0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal\" (UID: \"df743b01d5059c4d74a994cdac691aa0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.100241 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.100219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e160140d4aff5bc366703726e04ba72a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-198.ec2.internal\" (UID: \"e160140d4aff5bc366703726e04ba72a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.200257 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.200216 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.200418 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.200329 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df743b01d5059c4d74a994cdac691aa0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal\" (UID: \"df743b01d5059c4d74a994cdac691aa0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.200418 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.200298 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df743b01d5059c4d74a994cdac691aa0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal\" (UID: \"df743b01d5059c4d74a994cdac691aa0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.200418 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.200398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df743b01d5059c4d74a994cdac691aa0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal\" (UID: \"df743b01d5059c4d74a994cdac691aa0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.200548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.200424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e160140d4aff5bc366703726e04ba72a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-198.ec2.internal\" (UID: \"e160140d4aff5bc366703726e04ba72a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.200548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.200458 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e160140d4aff5bc366703726e04ba72a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-198.ec2.internal\" (UID: \"e160140d4aff5bc366703726e04ba72a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.200548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.200470 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df743b01d5059c4d74a994cdac691aa0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal\" (UID: \"df743b01d5059c4d74a994cdac691aa0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.300903 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.300809 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.361974 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.361942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.367517 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.366570 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" Jul 20 13:01:30.401158 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.401133 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.501560 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.501527 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.602017 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.601963 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.702544 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.702514 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.743697 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.743674 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 13:01:30.744152 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.743793 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 13:01:30.803534 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.803510 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.832293 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.832246 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 12:56:30 +0000 UTC" deadline="2028-03-18 12:38:49.42657897 +0000 UTC" Jul 20 13:01:30.832293 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.832281 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14567h37m18.594300855s" Jul 20 13:01:30.903863 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:30.903813 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:30.993419 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:30.993393 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 13:01:31.001076 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.001054 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 13:01:31.002293 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.002276 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientMemory" Jul 20 13:01:31.002372 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.002307 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 13:01:31.002372 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.002318 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeHasSufficientPID" Jul 20 13:01:31.004422 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.004403 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:31.015667 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:31.015635 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf743b01d5059c4d74a994cdac691aa0.slice/crio-442798025eeac3569f3e1016c73b461fa95222c80b629687d86dc6cb1ee5671c WatchSource:0}: Error finding container 442798025eeac3569f3e1016c73b461fa95222c80b629687d86dc6cb1ee5671c: Status 404 returned error can't find the container with id 442798025eeac3569f3e1016c73b461fa95222c80b629687d86dc6cb1ee5671c Jul 20 13:01:31.015854 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:31.015826 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode160140d4aff5bc366703726e04ba72a.slice/crio-19368dfc4a4b391f4a0e8a608978c4a8c48b9dab1065c01dadaf808b8fff7c14 WatchSource:0}: Error finding container 19368dfc4a4b391f4a0e8a608978c4a8c48b9dab1065c01dadaf808b8fff7c14: Status 404 returned error can't find the container with id 19368dfc4a4b391f4a0e8a608978c4a8c48b9dab1065c01dadaf808b8fff7c14 Jul 20 13:01:31.020218 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.020204 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 13:01:31.104887 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.104840 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:31.205185 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.205167 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-198.ec2.internal\" not found" Jul 20 13:01:31.277510 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.277476 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 13:01:31.298933 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.298901 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" Jul 20 13:01:31.315677 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.315655 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 13:01:31.317568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.317554 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" Jul 20 13:01:31.324842 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.324829 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 13:01:31.361796 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.361773 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 13:01:31.782800 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.782767 2564 apiserver.go:52] "Watching apiserver" Jul 20 13:01:31.791262 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.791228 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 13:01:31.793449 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.793422 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-qhscq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal","openshift-multus/network-metrics-daemon-ss8lq","openshift-network-operator/iptables-alerter-zlwdq","openshift-ovn-kubernetes/ovnkube-node-rjm7q","kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw","openshift-cluster-node-tuning-operator/tuned-2wpl6","openshift-dns/node-resolver-6klgv","openshift-image-registry/node-ca-c6kfj","openshift-multus/multus-7lvr4","openshift-multus/multus-additional-cni-plugins-qff7p","openshift-network-diagnostics/network-check-target-cwqfj","kube-system/global-pull-secret-syncer-xcmtb"] Jul 20 13:01:31.794839 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.794818 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.796547 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.796523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.797670 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.797652 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:31.797766 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.797731 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:31.797931 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.797902 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-572bd\"" Jul 20 13:01:31.798521 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.798502 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 13:01:31.798637 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.798615 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.798736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.798615 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.798794 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.798740 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 13:01:31.799063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.799045 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.799421 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.799399 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.799532 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.799449 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 13:01:31.799532 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.799412 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 13:01:31.799680 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.799631 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7qrsr\"" Jul 20 13:01:31.799680 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.799641 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.800773 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.800750 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.802293 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.802092 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.802293 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.802200 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 13:01:31.802480 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.802462 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s54h4\"" Jul 20 13:01:31.802582 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.802567 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.802713 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.802685 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.803561 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.803146 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.803561 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.803197 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.803561 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.803540 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gtcxh\"" Jul 20 13:01:31.803742 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.803671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:31.804547 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.804527 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 13:01:31.805117 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.805011 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cc5jg\"" Jul 20 13:01:31.805483 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.805458 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:31.805578 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.805533 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:31.805707 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.805688 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 13:01:31.805932 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.805916 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mp8gl\"" Jul 20 13:01:31.806141 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806126 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.806209 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806129 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.806608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-lib-modules\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.806688 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-device-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.806688 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806649 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb4ld\" (UniqueName: \"kubernetes.io/projected/4af5f1fe-975f-428f-800d-81e8e51810d7-kube-api-access-tb4ld\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.806688 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-cnibin\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.806842 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-os-release\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.806842 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806725 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/080c0903-adad-4f79-ac46-d12d29c75d63-cni-binary-copy\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.806842 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-hostroot\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.806842 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-var-lib-kubelet\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-socket-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.807046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806905 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:31.807046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/080c0903-adad-4f79-ac46-d12d29c75d63-multus-daemon-config\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.807046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lppr\" (UniqueName: \"kubernetes.io/projected/080c0903-adad-4f79-ac46-d12d29c75d63-kube-api-access-5lppr\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.807046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.806976 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9ada807-e503-449c-8224-6f75f60d04ef-host-slash\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.807046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-run\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-tuned\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-k8s-cni-cncf-io\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-netns\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysctl-conf\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-host\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807275 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-tmp\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807357 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2jqp\" (UniqueName: \"kubernetes.io/projected/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-kube-api-access-s2jqp\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807396 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-system-cni-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-conf-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807441 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysconfig\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-systemd\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807505 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-sys-fs\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807530 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8kwc\" (UniqueName: \"kubernetes.io/projected/d9ada807-e503-449c-8224-6f75f60d04ef-kube-api-access-j8kwc\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysctl-d\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-sys\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.807641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-cni-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsscb\" (UniqueName: \"kubernetes.io/projected/aa6bd398-f692-4fac-959b-f296c7ac6a30-kube-api-access-gsscb\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9ada807-e503-449c-8224-6f75f60d04ef-iptables-alerter-script\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807707 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-kubernetes\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-cni-bin\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807777 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-kubelet\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-etc-kubernetes\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-modprobe-d\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807898 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-registration-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-socket-dir-parent\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-cni-multus\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.807969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-multus-certs\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.808763 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.808301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:31.809474 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.809450 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ms8mh\"" Jul 20 13:01:31.810031 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.809933 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 13:01:31.810031 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.809954 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 13:01:31.811052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.810337 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.812820 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.812800 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 13:01:31.813099 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.813079 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.813396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.813374 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dmqz2\"" Jul 20 13:01:31.813574 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.813555 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xxmq2\"" Jul 20 13:01:31.813676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.813659 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 13:01:31.813721 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.813706 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.813863 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.813849 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 13:01:31.814178 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.814162 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 13:01:31.814585 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.814565 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 13:01:31.815163 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.815141 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 13:01:31.815492 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.815477 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 13:01:31.815764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.815744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:31.815922 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.815866 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:31.817047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.817027 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 13:01:31.832222 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.832202 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gd6ql" Jul 20 13:01:31.840891 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.840859 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gd6ql" Jul 20 13:01:31.900397 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.900374 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 13:01:31.908509 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908492 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9ada807-e503-449c-8224-6f75f60d04ef-iptables-alerter-script\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.908600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908517 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-kubernetes\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.908600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.908600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-cni-bin\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.908600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-systemd-units\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908625 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-kubernetes\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908636 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-cni-bin\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908639 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-log-socket\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908637 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908692 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-modprobe-d\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-registration-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.908765 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-socket-dir-parent\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-registration-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908805 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65a6fc4f-a16e-45bf-80c5-bb971c414db4-tmp-dir\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908840 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-modprobe-d\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-cni-netd\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908864 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovnkube-script-lib\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-socket-dir-parent\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908902 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15d906bc-6eff-4da3-a415-2621ca34552b-konnectivity-ca\") pod \"konnectivity-agent-qhscq\" (UID: \"15d906bc-6eff-4da3-a415-2621ca34552b\") " pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908921 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-lib-modules\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908935 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-device-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908956 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb4ld\" (UniqueName: \"kubernetes.io/projected/4af5f1fe-975f-428f-800d-81e8e51810d7-kube-api-access-tb4ld\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-cnibin\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.908988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-hostroot\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15d906bc-6eff-4da3-a415-2621ca34552b-agent-certs\") pod \"konnectivity-agent-qhscq\" (UID: \"15d906bc-6eff-4da3-a415-2621ca34552b\") " pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-device-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-host\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:31.909307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-hostroot\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-var-lib-kubelet\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lppr\" (UniqueName: \"kubernetes.io/projected/080c0903-adad-4f79-ac46-d12d29c75d63-kube-api-access-5lppr\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909110 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-var-lib-kubelet\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-cnibin\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-run-netns\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9ada807-e503-449c-8224-6f75f60d04ef-iptables-alerter-script\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-lib-modules\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-var-lib-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909190 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-env-overrides\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3791d792-594b-43fa-a1af-3ec0bf0aac2d-dbus\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-run\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909334 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-tuned\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909353 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-netns\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-ovn\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-run\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysctl-conf\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-tmp\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909461 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2jqp\" (UniqueName: \"kubernetes.io/projected/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-kube-api-access-s2jqp\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909521 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-etc-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909533 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-netns\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysconfig\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909585 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysctl-conf\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8kwc\" (UniqueName: \"kubernetes.io/projected/d9ada807-e503-449c-8224-6f75f60d04ef-kube-api-access-j8kwc\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysconfig\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909656 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-sys\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-cni-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909752 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-sys\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909741 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-node-log\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kqg2\" (UniqueName: \"kubernetes.io/projected/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-kube-api-access-7kqg2\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-kubelet\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-etc-kubernetes\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.910755 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909859 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-cni-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-os-release\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-etc-kubernetes\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-cni-multus\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-kubelet\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909962 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-multus-certs\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.909997 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-multus-certs\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-var-lib-cni-multus\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910025 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/65a6fc4f-a16e-45bf-80c5-bb971c414db4-hosts-file\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovnkube-config\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910100 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-os-release\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/080c0903-adad-4f79-ac46-d12d29c75d63-cni-binary-copy\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910142 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910157 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-slash\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-socket-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910199 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/080c0903-adad-4f79-ac46-d12d29c75d63-multus-daemon-config\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.911571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910233 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xcr4\" (UniqueName: \"kubernetes.io/projected/65a6fc4f-a16e-45bf-80c5-bb971c414db4-kube-api-access-4xcr4\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-os-release\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9ada807-e503-449c-8224-6f75f60d04ef-host-slash\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910296 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9ada807-e503-449c-8224-6f75f60d04ef-host-slash\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910288 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-socket-dir\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-k8s-cni-cncf-io\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910342 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-host-run-k8s-cni-cncf-io\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910380 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-system-cni-dir\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910424 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcfq6\" (UniqueName: \"kubernetes.io/projected/70fcd13b-8c85-4803-aabc-93c759f1f8d7-kube-api-access-xcfq6\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-kubelet\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910523 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.910537 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-host\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:31.910637 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:01:32.410615863 +0000 UTC m=+2.962508557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-host\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-system-cni-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-conf-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910723 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910785 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-multus-conf-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrmr7\" (UniqueName: \"kubernetes.io/projected/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-kube-api-access-vrmr7\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910849 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/080c0903-adad-4f79-ac46-d12d29c75d63-system-cni-dir\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910849 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/080c0903-adad-4f79-ac46-d12d29c75d63-multus-daemon-config\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910855 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-systemd\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910914 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-systemd\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910932 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-sys-fs\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.910959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-cni-bin\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovn-node-metrics-cert\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-sys-fs\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-serviceca\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3791d792-594b-43fa-a1af-3ec0bf0aac2d-kubelet-config\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/080c0903-adad-4f79-ac46-d12d29c75d63-cni-binary-copy\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysctl-d\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911207 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-sysctl-d\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.912568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.913054 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911243 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsscb\" (UniqueName: \"kubernetes.io/projected/aa6bd398-f692-4fac-959b-f296c7ac6a30-kube-api-access-gsscb\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:31.913054 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cnibin\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:31.913054 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911294 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4af5f1fe-975f-428f-800d-81e8e51810d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.913054 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.911320 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-systemd\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:31.913054 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.912852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-etc-tuned\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.913054 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.912838 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-tmp\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.918074 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.918016 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8kwc\" (UniqueName: \"kubernetes.io/projected/d9ada807-e503-449c-8224-6f75f60d04ef-kube-api-access-j8kwc\") pod \"iptables-alerter-zlwdq\" (UID: \"d9ada807-e503-449c-8224-6f75f60d04ef\") " pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:31.918173 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.918094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2jqp\" (UniqueName: \"kubernetes.io/projected/5df9b0b8-f41a-478b-97f7-d3cd2d3a2744-kube-api-access-s2jqp\") pod \"tuned-2wpl6\" (UID: \"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744\") " pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:31.918173 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.918088 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb4ld\" (UniqueName: \"kubernetes.io/projected/4af5f1fe-975f-428f-800d-81e8e51810d7-kube-api-access-tb4ld\") pod \"aws-ebs-csi-driver-node-nzqvw\" (UID: \"4af5f1fe-975f-428f-800d-81e8e51810d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:31.918173 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.918103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lppr\" (UniqueName: \"kubernetes.io/projected/080c0903-adad-4f79-ac46-d12d29c75d63-kube-api-access-5lppr\") pod \"multus-7lvr4\" (UID: \"080c0903-adad-4f79-ac46-d12d29c75d63\") " pod="openshift-multus/multus-7lvr4" Jul 20 13:01:31.923697 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.923680 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsscb\" (UniqueName: \"kubernetes.io/projected/aa6bd398-f692-4fac-959b-f296c7ac6a30-kube-api-access-gsscb\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:31.935310 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.935157 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" event={"ID":"df743b01d5059c4d74a994cdac691aa0","Type":"ContainerStarted","Data":"442798025eeac3569f3e1016c73b461fa95222c80b629687d86dc6cb1ee5671c"} Jul 20 13:01:31.937181 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:31.937155 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" event={"ID":"e160140d4aff5bc366703726e04ba72a","Type":"ContainerStarted","Data":"19368dfc4a4b391f4a0e8a608978c4a8c48b9dab1065c01dadaf808b8fff7c14"} Jul 20 13:01:32.012518 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/65a6fc4f-a16e-45bf-80c5-bb971c414db4-hosts-file\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.012518 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.012736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012535 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovnkube-config\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.012736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012592 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.012736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012603 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/65a6fc4f-a16e-45bf-80c5-bb971c414db4-hosts-file\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.012736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.012736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-slash\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.012736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xcr4\" (UniqueName: \"kubernetes.io/projected/65a6fc4f-a16e-45bf-80c5-bb971c414db4-kube-api-access-4xcr4\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-system-cni-dir\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfq6\" (UniqueName: \"kubernetes.io/projected/70fcd13b-8c85-4803-aabc-93c759f1f8d7-kube-api-access-xcfq6\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-kubelet\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012814 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-slash\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012901 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrmr7\" (UniqueName: \"kubernetes.io/projected/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-kube-api-access-vrmr7\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-cni-bin\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovn-node-metrics-cert\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-serviceca\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.012992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3791d792-594b-43fa-a1af-3ec0bf0aac2d-kubelet-config\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-system-cni-dir\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013020 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cnibin\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-kubelet\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013044 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-systemd\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013094 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013201 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cnibin\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013251 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3791d792-594b-43fa-a1af-3ec0bf0aac2d-kubelet-config\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovnkube-config\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013301 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-systemd\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013309 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-cni-bin\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013328 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013376 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-systemd-units\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-log-socket\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65a6fc4f-a16e-45bf-80c5-bb971c414db4-tmp-dir\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-cni-netd\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013505 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.013689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovnkube-script-lib\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15d906bc-6eff-4da3-a415-2621ca34552b-konnectivity-ca\") pod \"konnectivity-agent-qhscq\" (UID: \"15d906bc-6eff-4da3-a415-2621ca34552b\") " pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-systemd-units\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013570 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-log-socket\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15d906bc-6eff-4da3-a415-2621ca34552b-agent-certs\") pod \"konnectivity-agent-qhscq\" (UID: \"15d906bc-6eff-4da3-a415-2621ca34552b\") " pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013558 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-cni-netd\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013607 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-serviceca\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013607 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-host\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013662 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-run-netns\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-var-lib-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013710 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-env-overrides\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013724 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-host\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-var-lib-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3791d792-594b-43fa-a1af-3ec0bf0aac2d-dbus\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013785 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3791d792-594b-43fa-a1af-3ec0bf0aac2d-dbus\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-ovn\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-run-netns\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.014404 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013823 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-etc-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-node-log\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013922 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kqg2\" (UniqueName: \"kubernetes.io/projected/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-kube-api-access-7kqg2\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-os-release\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-etc-openvswitch\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.013952 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-run-ovn\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014038 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovnkube-script-lib\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/70fcd13b-8c85-4803-aabc-93c759f1f8d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.014090 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014146 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-env-overrides\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.014164 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:01:32.514147773 +0000 UTC m=+3.066040452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15d906bc-6eff-4da3-a415-2621ca34552b-konnectivity-ca\") pod \"konnectivity-agent-qhscq\" (UID: \"15d906bc-6eff-4da3-a415-2621ca34552b\") " pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/70fcd13b-8c85-4803-aabc-93c759f1f8d7-os-release\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-node-log\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.015004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.014294 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65a6fc4f-a16e-45bf-80c5-bb971c414db4-tmp-dir\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.015602 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.015574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-ovn-node-metrics-cert\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.016051 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.016037 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15d906bc-6eff-4da3-a415-2621ca34552b-agent-certs\") pod \"konnectivity-agent-qhscq\" (UID: \"15d906bc-6eff-4da3-a415-2621ca34552b\") " pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:32.021014 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.020992 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:01:32.021014 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.021010 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:01:32.021014 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.021019 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:32.021228 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.021060 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:01:32.521048168 +0000 UTC m=+3.072940845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:32.021739 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.021720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xcr4\" (UniqueName: \"kubernetes.io/projected/65a6fc4f-a16e-45bf-80c5-bb971c414db4-kube-api-access-4xcr4\") pod \"node-resolver-6klgv\" (UID: \"65a6fc4f-a16e-45bf-80c5-bb971c414db4\") " pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.021838 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.021752 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrmr7\" (UniqueName: \"kubernetes.io/projected/0a87dc87-258f-47ea-b5a4-1e5656f6a80c-kube-api-access-vrmr7\") pod \"node-ca-c6kfj\" (UID: \"0a87dc87-258f-47ea-b5a4-1e5656f6a80c\") " pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.023127 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.023109 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcfq6\" (UniqueName: \"kubernetes.io/projected/70fcd13b-8c85-4803-aabc-93c759f1f8d7-kube-api-access-xcfq6\") pod \"multus-additional-cni-plugins-qff7p\" (UID: \"70fcd13b-8c85-4803-aabc-93c759f1f8d7\") " pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.024427 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.024412 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kqg2\" (UniqueName: \"kubernetes.io/projected/88320fe0-e206-431b-ac3e-8ab93c4ab9ac-kube-api-access-7kqg2\") pod \"ovnkube-node-rjm7q\" (UID: \"88320fe0-e206-431b-ac3e-8ab93c4ab9ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.106783 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.106751 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" Jul 20 13:01:32.116318 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.116298 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7lvr4" Jul 20 13:01:32.116831 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.116808 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af5f1fe_975f_428f_800d_81e8e51810d7.slice/crio-1cb7f3208ebcf0ebf3d960ac4a1e0c4c452d02fdd9d6c72c1c0ffa49df2c98d7 WatchSource:0}: Error finding container 1cb7f3208ebcf0ebf3d960ac4a1e0c4c452d02fdd9d6c72c1c0ffa49df2c98d7: Status 404 returned error can't find the container with id 1cb7f3208ebcf0ebf3d960ac4a1e0c4c452d02fdd9d6c72c1c0ffa49df2c98d7 Jul 20 13:01:32.124574 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.124552 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080c0903_adad_4f79_ac46_d12d29c75d63.slice/crio-b152783be201a440e8dad1dcf88a32eaaefe8149be1901fe68c1539cd3d89ade WatchSource:0}: Error finding container b152783be201a440e8dad1dcf88a32eaaefe8149be1901fe68c1539cd3d89ade: Status 404 returned error can't find the container with id b152783be201a440e8dad1dcf88a32eaaefe8149be1901fe68c1539cd3d89ade Jul 20 13:01:32.125680 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.125656 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zlwdq" Jul 20 13:01:32.131296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.131280 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" Jul 20 13:01:32.132986 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.132966 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ada807_e503_449c_8224_6f75f60d04ef.slice/crio-b19c84092137befc2a41ad9fe0a695f694e75f7fe76c2539433d29cc9190632d WatchSource:0}: Error finding container b19c84092137befc2a41ad9fe0a695f694e75f7fe76c2539433d29cc9190632d: Status 404 returned error can't find the container with id b19c84092137befc2a41ad9fe0a695f694e75f7fe76c2539433d29cc9190632d Jul 20 13:01:32.138100 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.138081 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qff7p" Jul 20 13:01:32.138359 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.138330 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df9b0b8_f41a_478b_97f7_d3cd2d3a2744.slice/crio-708c3f3d7f986484dcb9feb6418b50af4bcd5a3f2425502dfad06a9950035ae5 WatchSource:0}: Error finding container 708c3f3d7f986484dcb9feb6418b50af4bcd5a3f2425502dfad06a9950035ae5: Status 404 returned error can't find the container with id 708c3f3d7f986484dcb9feb6418b50af4bcd5a3f2425502dfad06a9950035ae5 Jul 20 13:01:32.145545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.145524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6klgv" Jul 20 13:01:32.152814 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.152774 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:32.154668 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.154641 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a6fc4f_a16e_45bf_80c5_bb971c414db4.slice/crio-4b73e9e3e6378429fda12449c4bcef2812b3ada1bce95b08d84c838773ac8c51 WatchSource:0}: Error finding container 4b73e9e3e6378429fda12449c4bcef2812b3ada1bce95b08d84c838773ac8c51: Status 404 returned error can't find the container with id 4b73e9e3e6378429fda12449c4bcef2812b3ada1bce95b08d84c838773ac8c51 Jul 20 13:01:32.161119 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.161094 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c6kfj" Jul 20 13:01:32.161923 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.161872 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d906bc_6eff_4da3_a415_2621ca34552b.slice/crio-9fc9f61776701e168b0e0f3530f9b91132ed9ac4e8848f65fa77ef16f8420c72 WatchSource:0}: Error finding container 9fc9f61776701e168b0e0f3530f9b91132ed9ac4e8848f65fa77ef16f8420c72: Status 404 returned error can't find the container with id 9fc9f61776701e168b0e0f3530f9b91132ed9ac4e8848f65fa77ef16f8420c72 Jul 20 13:01:32.166269 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.166245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:01:32.171499 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.171473 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a87dc87_258f_47ea_b5a4_1e5656f6a80c.slice/crio-7d75091c07bb25c4c7201a032083dd9aa9a6baa40ea6b2b9c1b5ec2df6d2b3e4 WatchSource:0}: Error finding container 7d75091c07bb25c4c7201a032083dd9aa9a6baa40ea6b2b9c1b5ec2df6d2b3e4: Status 404 returned error can't find the container with id 7d75091c07bb25c4c7201a032083dd9aa9a6baa40ea6b2b9c1b5ec2df6d2b3e4 Jul 20 13:01:32.176563 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:01:32.176540 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88320fe0_e206_431b_ac3e_8ab93c4ab9ac.slice/crio-9113221149b2249a105bc99e0b50ea09379b2ecbb2ec753b3d3486f33abcb2d3 WatchSource:0}: Error finding container 9113221149b2249a105bc99e0b50ea09379b2ecbb2ec753b3d3486f33abcb2d3: Status 404 returned error can't find the container with id 9113221149b2249a105bc99e0b50ea09379b2ecbb2ec753b3d3486f33abcb2d3 Jul 20 13:01:32.205880 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.205858 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 13:01:32.417544 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.417446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:32.417708 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.417641 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:32.417777 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.417724 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:01:33.417702327 +0000 UTC m=+3.969595025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:32.518629 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.518583 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:32.518826 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.518781 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:32.518926 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.518881 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:01:33.518846812 +0000 UTC m=+4.070739511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:32.619050 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.619018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:32.619192 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.619157 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:01:32.619192 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.619172 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:01:32.619192 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.619182 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:32.619302 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:32.619234 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:01:33.619220964 +0000 UTC m=+4.171113642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:32.843626 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.843535 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 12:56:32 +0000 UTC" deadline="2028-01-29 18:27:22.284911284 +0000 UTC" Jul 20 13:01:32.843626 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.843574 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13397h25m49.441342404s" Jul 20 13:01:32.954006 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.952545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6klgv" event={"ID":"65a6fc4f-a16e-45bf-80c5-bb971c414db4","Type":"ContainerStarted","Data":"4b73e9e3e6378429fda12449c4bcef2812b3ada1bce95b08d84c838773ac8c51"} Jul 20 13:01:32.958079 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.958048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" event={"ID":"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744","Type":"ContainerStarted","Data":"708c3f3d7f986484dcb9feb6418b50af4bcd5a3f2425502dfad06a9950035ae5"} Jul 20 13:01:32.964830 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.964730 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"9113221149b2249a105bc99e0b50ea09379b2ecbb2ec753b3d3486f33abcb2d3"} Jul 20 13:01:32.968209 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.968137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c6kfj" event={"ID":"0a87dc87-258f-47ea-b5a4-1e5656f6a80c","Type":"ContainerStarted","Data":"7d75091c07bb25c4c7201a032083dd9aa9a6baa40ea6b2b9c1b5ec2df6d2b3e4"} Jul 20 13:01:32.973047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.972944 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerStarted","Data":"6c2fc78d47c45ebed9a72bb8df97b5aa4561b53d9b617b91a0d405d63ff02072"} Jul 20 13:01:32.983398 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.983271 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zlwdq" event={"ID":"d9ada807-e503-449c-8224-6f75f60d04ef","Type":"ContainerStarted","Data":"b19c84092137befc2a41ad9fe0a695f694e75f7fe76c2539433d29cc9190632d"} Jul 20 13:01:32.986713 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.986634 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7lvr4" event={"ID":"080c0903-adad-4f79-ac46-d12d29c75d63","Type":"ContainerStarted","Data":"b152783be201a440e8dad1dcf88a32eaaefe8149be1901fe68c1539cd3d89ade"} Jul 20 13:01:32.989996 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.989908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" event={"ID":"4af5f1fe-975f-428f-800d-81e8e51810d7","Type":"ContainerStarted","Data":"1cb7f3208ebcf0ebf3d960ac4a1e0c4c452d02fdd9d6c72c1c0ffa49df2c98d7"} Jul 20 13:01:32.993989 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.993721 2564 generic.go:356] "Generic (PLEG): container finished" podID="df743b01d5059c4d74a994cdac691aa0" containerID="d70c797cd77b59aa6c30cab6b7a6e3688f15e4d965f2eb6534da245c76c15b1d" exitCode=0 Jul 20 13:01:32.993989 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:32.993799 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" event={"ID":"df743b01d5059c4d74a994cdac691aa0","Type":"ContainerDied","Data":"d70c797cd77b59aa6c30cab6b7a6e3688f15e4d965f2eb6534da245c76c15b1d"} Jul 20 13:01:33.000676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.000603 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhscq" event={"ID":"15d906bc-6eff-4da3-a415-2621ca34552b","Type":"ContainerStarted","Data":"9fc9f61776701e168b0e0f3530f9b91132ed9ac4e8848f65fa77ef16f8420c72"} Jul 20 13:01:33.425734 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.425692 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:33.425932 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.425891 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:33.426039 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.425979 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:01:35.425959807 +0000 UTC m=+5.977852487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:33.526757 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.526108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:33.526757 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.526299 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:33.526757 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.526364 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:01:35.526345248 +0000 UTC m=+6.078237953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:33.627377 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.627339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:33.627552 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.627523 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:01:33.627552 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.627543 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:01:33.627689 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.627557 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:33.627689 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.627615 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:01:35.627597099 +0000 UTC m=+6.179489777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:33.844162 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.844044 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 12:56:32 +0000 UTC" deadline="2028-02-01 11:16:54.458595189 +0000 UTC" Jul 20 13:01:33.844162 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.844109 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13462h15m20.614490221s" Jul 20 13:01:33.932164 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.931079 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:33.932164 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.931207 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:33.932164 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.931611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:33.932164 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.931719 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:33.932783 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:33.932613 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:33.932783 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:33.932724 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:34.013382 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:34.013340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" event={"ID":"df743b01d5059c4d74a994cdac691aa0","Type":"ContainerStarted","Data":"977e851d8ecaa6ba67977e4142a5a7d26bb70f2e0e5ca7bc625c97d0a7e3a33a"} Jul 20 13:01:34.034187 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:34.034129 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-198.ec2.internal" podStartSLOduration=3.034108853 podStartE2EDuration="3.034108853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:34.033053343 +0000 UTC m=+4.584946074" watchObservedRunningTime="2026-07-20 13:01:34.034108853 +0000 UTC m=+4.586001556" Jul 20 13:01:35.446745 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:35.446689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:35.447267 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.446857 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:35.447267 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.446952 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:01:39.446929107 +0000 UTC m=+9.998821806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:35.547311 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:35.547263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:35.547516 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.547448 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:35.547586 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.547524 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:01:39.547509262 +0000 UTC m=+10.099401941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:35.648440 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:35.648395 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:35.648623 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.648571 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:01:35.648623 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.648592 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:01:35.648623 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.648618 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:35.648798 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.648685 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:01:39.648664507 +0000 UTC m=+10.200557190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:35.934781 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:35.934214 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:35.934781 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:35.934259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:35.934781 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.934366 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:35.934781 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:35.934454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:35.934781 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.934556 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:35.934781 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:35.934624 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:37.929784 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:37.929742 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:37.930298 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:37.929745 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:37.930298 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:37.929935 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:37.930298 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:37.929889 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:37.930298 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:37.930101 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:37.930298 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:37.930189 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:39.480544 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:39.480503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:39.481004 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.480648 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:39.481004 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.480715 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:01:47.480695787 +0000 UTC m=+18.032588472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:39.582005 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:39.581789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:39.582177 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.582019 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:39.582177 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.582087 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:01:47.582068153 +0000 UTC m=+18.133960832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:39.683187 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:39.683119 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:39.683379 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.683323 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:01:39.683379 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.683349 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:01:39.683379 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.683365 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:39.683554 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.683424 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:01:47.683405217 +0000 UTC m=+18.235297897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:39.931781 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:39.931692 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:39.931968 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.931830 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:39.931968 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:39.931928 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:39.932097 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.932045 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:39.932097 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:39.932093 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:39.932236 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:39.932170 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:41.930163 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:41.930133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:41.930598 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:41.930135 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:41.930598 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:41.930250 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:41.930598 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:41.930271 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:41.930598 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:41.930376 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:41.930598 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:41.930448 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:43.930853 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:43.930619 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:43.931297 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:43.930618 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:43.931297 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:43.930976 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:43.931297 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:43.931047 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:43.931297 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:43.930618 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:43.931297 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:43.931123 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:44.037710 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.037671 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c6kfj" event={"ID":"0a87dc87-258f-47ea-b5a4-1e5656f6a80c","Type":"ContainerStarted","Data":"b95a3aa94d9e6fc1c322ddbb595857c818d7f46e30807ca5cd5648a55bc33ec0"} Jul 20 13:01:44.039691 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.039656 2564 generic.go:356] "Generic (PLEG): container finished" podID="70fcd13b-8c85-4803-aabc-93c759f1f8d7" containerID="d0bf30a10bf6ead66612567ebdcffae9cd48ccfdae26a1587ab95be947ae6385" exitCode=0 Jul 20 13:01:44.039824 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.039747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerDied","Data":"d0bf30a10bf6ead66612567ebdcffae9cd48ccfdae26a1587ab95be947ae6385"} Jul 20 13:01:44.041497 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.041473 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" event={"ID":"4af5f1fe-975f-428f-800d-81e8e51810d7","Type":"ContainerStarted","Data":"5ab8fbb0c08d928876c5891e930b9f13b642cb2dbc9290d09ae8b8816a906353"} Jul 20 13:01:44.043301 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.043274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhscq" event={"ID":"15d906bc-6eff-4da3-a415-2621ca34552b","Type":"ContainerStarted","Data":"3ce200152ff4485bedd58459d1f916904662972da2dd84833a95cc0d6fb05c44"} Jul 20 13:01:44.044949 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.044925 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6klgv" event={"ID":"65a6fc4f-a16e-45bf-80c5-bb971c414db4","Type":"ContainerStarted","Data":"5cf738872c34dc9a8ef342b5304c9f271c2b68cb9daf2e193b44cc0dce6c78a7"} Jul 20 13:01:44.046519 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.046499 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" event={"ID":"5df9b0b8-f41a-478b-97f7-d3cd2d3a2744","Type":"ContainerStarted","Data":"718699754a0037bbe235066bc954eac3a00fce8facca75ddd9c411344cc20661"} Jul 20 13:01:44.052423 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.052375 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c6kfj" podStartSLOduration=2.156669293 podStartE2EDuration="13.052362422s" totalImagesPullingTime="10.895693129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:44.051892772 +0000 UTC m=+14.603785496" watchObservedRunningTime="2026-07-20 13:01:44.052362422 +0000 UTC m=+14.604255120" Jul 20 13:01:44.069234 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.069187 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2wpl6" podStartSLOduration=3.108623694 podStartE2EDuration="14.069168803s" totalImagesPullingTime="10.960545109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:44.068379998 +0000 UTC m=+14.620272699" watchObservedRunningTime="2026-07-20 13:01:44.069168803 +0000 UTC m=+14.621061506" Jul 20 13:01:44.149254 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.149152 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6klgv" podStartSLOduration=3.23607756 podStartE2EDuration="14.149130535s" totalImagesPullingTime="10.913052975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:44.14593666 +0000 UTC m=+14.697829390" watchObservedRunningTime="2026-07-20 13:01:44.149130535 +0000 UTC m=+14.701023235" Jul 20 13:01:44.177416 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:44.177366 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qhscq" podStartSLOduration=3.32226769 podStartE2EDuration="14.177345703s" totalImagesPullingTime="10.855078013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:44.177154601 +0000 UTC m=+14.729047301" watchObservedRunningTime="2026-07-20 13:01:44.177345703 +0000 UTC m=+14.729238406" Jul 20 13:01:45.051013 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:45.050966 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zlwdq" event={"ID":"d9ada807-e503-449c-8224-6f75f60d04ef","Type":"ContainerStarted","Data":"0800c891d97c58037605c41789035a46c66e7d9c16de9c9917c323b6854f87e3"} Jul 20 13:01:45.070655 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:45.070599 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zlwdq" podStartSLOduration=4.135800971 podStartE2EDuration="15.07058413s" totalImagesPullingTime="10.934783159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:45.069656172 +0000 UTC m=+15.621548871" watchObservedRunningTime="2026-07-20 13:01:45.07058413 +0000 UTC m=+15.622476828" Jul 20 13:01:45.929734 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:45.929701 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:45.929734 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:45.929701 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:45.929990 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:45.929702 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:45.930118 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:45.929902 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:45.930118 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:45.930077 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:45.930118 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:45.930074 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:46.899810 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:46.899766 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:46.900490 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:46.900370 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:47.544573 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:47.544536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:47.544756 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.544710 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:47.544815 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.544804 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:03.544781628 +0000 UTC m=+34.096674307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:01:47.645030 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:47.645001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:47.645204 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.645140 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:47.645204 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.645194 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:02:03.645181583 +0000 UTC m=+34.197074260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:01:47.745569 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:47.745542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:47.745748 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.745729 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:01:47.745802 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.745757 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:01:47.745802 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.745772 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:47.745911 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.745823 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:02:03.745809471 +0000 UTC m=+34.297702148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:01:47.929639 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:47.929557 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:47.930174 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:47.929656 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:47.930174 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.929663 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:47.930174 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.929775 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:47.930174 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:47.929825 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:47.930174 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:47.929921 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:49.931254 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:49.931196 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:49.931254 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:49.931239 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:49.931997 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:49.931286 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:49.931997 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:49.931339 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:49.931997 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:49.931394 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:49.931997 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:49.931462 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:51.930042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:51.930009 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:51.930042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:51.930027 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:51.930547 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:51.930027 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:51.930547 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:51.930140 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:51.930547 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:51.930200 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:51.930547 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:51.930274 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:53.929748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:53.929680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:53.929748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:53.929695 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:53.929748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:53.929693 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:53.930388 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:53.929805 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:53.930388 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:53.929869 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:53.930388 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:53.929972 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:54.666130 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:54.666105 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 13:01:54.879571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:54.879473 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T13:01:54.666124732Z","UUID":"07270afc-e2a2-4726-bc59-1f8ec4de662c","Handler":null,"Name":"","Endpoint":""} Jul 20 13:01:54.880805 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:54.880786 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 13:01:54.880915 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:54.880823 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 13:01:55.071812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.071775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" event={"ID":"4af5f1fe-975f-428f-800d-81e8e51810d7","Type":"ContainerStarted","Data":"8b0a4e58a605a32f3af06663232ac6df11f2ec085114188688fafe3d7474bcf2"} Jul 20 13:01:55.073179 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.073151 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" event={"ID":"e160140d4aff5bc366703726e04ba72a","Type":"ContainerStarted","Data":"f8f9df123816c46f737f5307fa1ee2dec068d48ba090cd572f4ed72f5e41659a"} Jul 20 13:01:55.075578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.075561 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:01:55.075885 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.075853 2564 generic.go:356] "Generic (PLEG): container finished" podID="88320fe0-e206-431b-ac3e-8ab93c4ab9ac" containerID="182b4b0a9f983d60fdd80e80031a392e45bda14b7bdf7b22e1ea2852b19286a6" exitCode=1 Jul 20 13:01:55.075940 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.075901 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"9a281e0754f34b8f7a096edaa42011b4e73fd232b10e36ac6f21f904a4b5d6c1"} Jul 20 13:01:55.075981 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.075955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"f847e29adbf8f7c0d1876191f4312cf8ce7f1bdf80b744e30c6233a682be2d73"} Jul 20 13:01:55.076018 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.075988 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"0f9d563bf64128924c66d560e65e84b5665374138ecdf9c1dd2fc44ac581e8bc"} Jul 20 13:01:55.076058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.076030 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"d94a0a1621ad4ee73d8269ed45413d41cc224908b4bd25d57d5e9a4c5834c8d9"} Jul 20 13:01:55.076094 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.076059 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerDied","Data":"182b4b0a9f983d60fdd80e80031a392e45bda14b7bdf7b22e1ea2852b19286a6"} Jul 20 13:01:55.076094 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.076088 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"8e780ebded6f5f4160a6bd6714dbfe4940f843e4d37716cd5760fb5fd102c595"} Jul 20 13:01:55.077598 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.077575 2564 generic.go:356] "Generic (PLEG): container finished" podID="70fcd13b-8c85-4803-aabc-93c759f1f8d7" containerID="3859f80aa22fb3a9d60d13706afd3ddfba518915237cc2ad4768cd8472da4ec8" exitCode=0 Jul 20 13:01:55.077691 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.077638 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerDied","Data":"3859f80aa22fb3a9d60d13706afd3ddfba518915237cc2ad4768cd8472da4ec8"} Jul 20 13:01:55.078897 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.078858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7lvr4" event={"ID":"080c0903-adad-4f79-ac46-d12d29c75d63","Type":"ContainerStarted","Data":"6779eb431c51a7ff200c022d53c46f433bf78764288b5c83db140f76fafe91e1"} Jul 20 13:01:55.089125 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.089088 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-198.ec2.internal" podStartSLOduration=24.089075995 podStartE2EDuration="24.089075995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:55.088481351 +0000 UTC m=+25.640374071" watchObservedRunningTime="2026-07-20 13:01:55.089075995 +0000 UTC m=+25.640968694" Jul 20 13:01:55.129739 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.129696 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7lvr4" podStartSLOduration=2.784807788 podStartE2EDuration="25.129681552s" totalImagesPullingTime="22.344873764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:55.129472133 +0000 UTC m=+25.681364831" watchObservedRunningTime="2026-07-20 13:01:55.129681552 +0000 UTC m=+25.681574247" Jul 20 13:01:55.930230 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.930204 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:55.930386 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.930206 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:55.930386 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:55.930308 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:55.930386 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:55.930318 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:55.930551 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:55.930427 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:55.930551 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:55.930529 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:56.083396 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:56.083364 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" event={"ID":"4af5f1fe-975f-428f-800d-81e8e51810d7","Type":"ContainerStarted","Data":"5f66fb80768b6fd8f227d295486ca8f1b0623d5735616f05e29eccc5b4607324"} Jul 20 13:01:56.104751 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:56.104689 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzqvw" podStartSLOduration=2.724612634 podStartE2EDuration="26.104673849s" totalImagesPullingTime="23.380061215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:01:56.104232794 +0000 UTC m=+26.656125518" watchObservedRunningTime="2026-07-20 13:01:56.104673849 +0000 UTC m=+26.656566547" Jul 20 13:01:57.087568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:57.087529 2564 generic.go:356] "Generic (PLEG): container finished" podID="70fcd13b-8c85-4803-aabc-93c759f1f8d7" containerID="2213fc858728f0f19f076ac489f3e9c41504009ad4151d8445ea738682bf1dcf" exitCode=0 Jul 20 13:01:57.088050 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:57.087609 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerDied","Data":"2213fc858728f0f19f076ac489f3e9c41504009ad4151d8445ea738682bf1dcf"} Jul 20 13:01:57.930131 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:57.930105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:57.930218 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:57.930142 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:57.930218 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:57.930109 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:57.930320 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:57.930224 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:57.930320 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:57.930287 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:01:57.930418 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:57.930356 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:58.092493 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:58.092418 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:01:58.092913 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:58.092794 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"38f742445a58eb1212b2e79cd3af82b0f25fdadf01fc23a5cfed0adcb11f5464"} Jul 20 13:01:58.095047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:58.095022 2564 generic.go:356] "Generic (PLEG): container finished" podID="70fcd13b-8c85-4803-aabc-93c759f1f8d7" containerID="692d2e2e36a0fe843ab44442237372ef3718d2109088e8e369f1de8a41b1c216" exitCode=0 Jul 20 13:01:58.095172 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:58.095075 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerDied","Data":"692d2e2e36a0fe843ab44442237372ef3718d2109088e8e369f1de8a41b1c216"} Jul 20 13:01:59.930248 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:59.930220 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:59.930272 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qhscq" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:59.930347 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:59.930444 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:59.930701 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:01:59.930776 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:59.930793 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:01:59.930933 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:01:59.930839 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:02:00.102751 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:00.102720 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:02:00.103173 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:00.103140 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"6ed9d00b10eb2829efe42f377d8a82d64ef9624d57ffec1fbf4725b9e97483fe"} Jul 20 13:02:00.103503 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:00.103438 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:02:00.103646 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:00.103623 2564 scope.go:117] "RemoveContainer" containerID="182b4b0a9f983d60fdd80e80031a392e45bda14b7bdf7b22e1ea2852b19286a6" Jul 20 13:02:00.123349 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:00.123326 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:02:01.110763 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.110731 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:02:01.111246 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.111189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" event={"ID":"88320fe0-e206-431b-ac3e-8ab93c4ab9ac","Type":"ContainerStarted","Data":"a1ffbc4fdd0a10741c7de48cf11f728d5ef33afbcd85bab8e3926343aaf8f7dc"} Jul 20 13:02:01.111429 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.111406 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:02:01.111530 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.111487 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:02:01.130618 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.130591 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:02:01.139470 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.139427 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" podStartSLOduration=9.613069765 podStartE2EDuration="30.139413482s" totalImagesPullingTime="20.526343717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:01.139360311 +0000 UTC m=+31.691253035" watchObservedRunningTime="2026-07-20 13:02:01.139413482 +0000 UTC m=+31.691306181" Jul 20 13:02:01.208533 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.208498 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xcmtb"] Jul 20 13:02:01.208704 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.208624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:01.208762 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:01.208735 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:02:01.216222 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.216148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cwqfj"] Jul 20 13:02:01.216368 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.216269 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:01.216429 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:01.216367 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:02:01.216944 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.216851 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ss8lq"] Jul 20 13:02:01.217069 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:01.216968 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:01.217135 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:01.217075 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:02:02.929752 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:02.929711 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:02.929752 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:02.929743 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:02.930272 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:02.929827 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:02.930272 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:02.929833 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:02:02.930272 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:02.929920 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:02:02.930272 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:02.930008 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:02:03.567663 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:03.567440 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:03.567824 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.567588 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:02:03.567824 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.567812 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs podName:aa6bd398-f692-4fac-959b-f296c7ac6a30 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:35.567792989 +0000 UTC m=+66.119685671 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs") pod "network-metrics-daemon-ss8lq" (UID: "aa6bd398-f692-4fac-959b-f296c7ac6a30") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 13:02:03.668888 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:03.668843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:03.669034 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.668990 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 13:02:03.669070 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.669053 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret podName:3791d792-594b-43fa-a1af-3ec0bf0aac2d nodeName:}" failed. No retries permitted until 2026-07-20 13:02:35.669038766 +0000 UTC m=+66.220931443 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret") pod "global-pull-secret-syncer-xcmtb" (UID: "3791d792-594b-43fa-a1af-3ec0bf0aac2d") : object "kube-system"/"original-pull-secret" not registered Jul 20 13:02:03.769369 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:03.769336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:03.769551 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.769531 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 13:02:03.769596 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.769561 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 13:02:03.769596 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.769575 2564 projected.go:196] Error preparing data for projected volume kube-api-access-7bjrg for pod openshift-network-diagnostics/network-check-target-cwqfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:02:03.769659 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:03.769640 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg podName:a90281c1-8b44-462e-8f2f-7d7eaf23210f nodeName:}" failed. No retries permitted until 2026-07-20 13:02:35.769620475 +0000 UTC m=+66.321513159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7bjrg" (UniqueName: "kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg") pod "network-check-target-cwqfj" (UID: "a90281c1-8b44-462e-8f2f-7d7eaf23210f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 13:02:04.121661 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:04.121627 2564 generic.go:356] "Generic (PLEG): container finished" podID="70fcd13b-8c85-4803-aabc-93c759f1f8d7" containerID="e8cda8135d819cb2f1eab11e525daae7b73d9588a39ee29993d48f9582c6a04b" exitCode=0 Jul 20 13:02:04.122235 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:04.121679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerDied","Data":"e8cda8135d819cb2f1eab11e525daae7b73d9588a39ee29993d48f9582c6a04b"} Jul 20 13:02:04.929614 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:04.929583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:04.929757 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:04.929583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:04.929757 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:04.929677 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:02:04.929757 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:04.929587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:04.929946 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:04.929765 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:02:04.929946 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:04.929859 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:02:05.126812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:05.126782 2564 generic.go:356] "Generic (PLEG): container finished" podID="70fcd13b-8c85-4803-aabc-93c759f1f8d7" containerID="6097236af9e5c7e2a2d373b02e2474cb7cbdb932885c4edbf585d875b394cb17" exitCode=0 Jul 20 13:02:05.127357 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:05.126832 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerDied","Data":"6097236af9e5c7e2a2d373b02e2474cb7cbdb932885c4edbf585d875b394cb17"} Jul 20 13:02:06.131970 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:06.131929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qff7p" event={"ID":"70fcd13b-8c85-4803-aabc-93c759f1f8d7","Type":"ContainerStarted","Data":"1618c1380993a2a931c814c98502991b63c3b1b5bd6c426eed457d4106d56281"} Jul 20 13:02:06.157572 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:06.157521 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qff7p" podStartSLOduration=7.465913108 podStartE2EDuration="36.157506426s" totalImagesPullingTime="28.691593318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:06.155983456 +0000 UTC m=+36.707876156" watchObservedRunningTime="2026-07-20 13:02:06.157506426 +0000 UTC m=+36.709399125" Jul 20 13:02:06.930215 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:06.930187 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:06.930330 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:06.930297 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:06.930330 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:06.930309 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xcmtb" podUID="3791d792-594b-43fa-a1af-3ec0bf0aac2d" Jul 20 13:02:06.930330 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:06.930328 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:06.930480 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:06.930410 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ss8lq" podUID="aa6bd398-f692-4fac-959b-f296c7ac6a30" Jul 20 13:02:06.930480 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:06.930470 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwqfj" podUID="a90281c1-8b44-462e-8f2f-7d7eaf23210f" Jul 20 13:02:07.225401 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.225377 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-198.ec2.internal" event="NodeReady" Jul 20 13:02:07.225843 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.225513 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 13:02:07.259368 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.259343 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql"] Jul 20 13:02:07.293941 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.293913 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jnvhs"] Jul 20 13:02:07.294093 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.294037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.298552 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.298119 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.298552 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.298158 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 20 13:02:07.298552 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.298161 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 20 13:02:07.298552 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.298428 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.299295 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.298904 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xldzf\"" Jul 20 13:02:07.309721 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.309699 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s"] Jul 20 13:02:07.309828 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.309774 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.312504 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.312479 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.312599 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.312484 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 20 13:02:07.312681 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.312645 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 20 13:02:07.312759 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.312684 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.312824 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.312775 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-llz6f\"" Jul 20 13:02:07.319688 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.319670 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 20 13:02:07.331679 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.331656 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm"] Jul 20 13:02:07.331797 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.331778 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" Jul 20 13:02:07.334143 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.334129 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.334294 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.334273 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zg79q\"" Jul 20 13:02:07.334386 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.334367 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.351702 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.351683 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d66f77fc9-zckp5"] Jul 20 13:02:07.351809 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.351793 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.354395 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.354363 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 20 13:02:07.354620 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.354603 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-btl28\"" Jul 20 13:02:07.354702 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.354637 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 20 13:02:07.354702 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.354697 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.355051 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.355036 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.368732 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.368713 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl"] Jul 20 13:02:07.368820 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.368811 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.371301 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.371286 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 13:02:07.371413 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.371320 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 13:02:07.371413 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.371328 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tpf8d\"" Jul 20 13:02:07.371413 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.371347 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 13:02:07.379814 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.379797 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 13:02:07.383018 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.382996 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5655955bbb-7fznj"] Jul 20 13:02:07.383132 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.383116 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.385776 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.385758 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.386955 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.386937 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.387036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.386974 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 20 13:02:07.387036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.387015 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vndj9\"" Jul 20 13:02:07.387128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.387119 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 20 13:02:07.397451 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50bbd98b-5610-4ad4-a413-3e4eec07581e-tmp\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.397557 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397461 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50bbd98b-5610-4ad4-a413-3e4eec07581e-service-ca-bundle\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.397557 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85eff64d-ea97-41ab-a625-39ae59200b81-config\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.397557 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397521 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50bbd98b-5610-4ad4-a413-3e4eec07581e-serving-cert\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.397694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397582 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bp9\" (UniqueName: \"kubernetes.io/projected/85eff64d-ea97-41ab-a625-39ae59200b81-kube-api-access-b9bp9\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.397694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/50bbd98b-5610-4ad4-a413-3e4eec07581e-snapshots\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.397694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50bbd98b-5610-4ad4-a413-3e4eec07581e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.397694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397680 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74nk6\" (UniqueName: \"kubernetes.io/projected/50bbd98b-5610-4ad4-a413-3e4eec07581e-kube-api-access-74nk6\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.397823 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.397707 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85eff64d-ea97-41ab-a625-39ae59200b81-serving-cert\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.403978 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.403961 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-nhh9k"] Jul 20 13:02:07.404083 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.404067 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.406607 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.406585 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 20 13:02:07.406708 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.406653 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 20 13:02:07.406708 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.406672 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.406811 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.406793 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nr2p9\"" Jul 20 13:02:07.406899 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.406867 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 20 13:02:07.406983 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.406967 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 20 13:02:07.407145 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.407130 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.427940 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.427919 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm"] Jul 20 13:02:07.428055 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.428041 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.430593 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.430576 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 13:02:07.430593 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.430587 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 13:02:07.430742 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.430578 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.430742 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.430578 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mnvxz\"" Jul 20 13:02:07.430742 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.430657 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.436652 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.436634 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 13:02:07.452076 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452056 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jnvhs"] Jul 20 13:02:07.452155 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452083 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql"] Jul 20 13:02:07.452155 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452097 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s"] Jul 20 13:02:07.452155 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452110 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm"] Jul 20 13:02:07.452155 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452126 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm"] Jul 20 13:02:07.452155 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452145 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wgngz"] Jul 20 13:02:07.452324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.452155 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:07.454836 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.454815 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 20 13:02:07.454953 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.454892 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-47p9x\"" Jul 20 13:02:07.454953 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.454892 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.454953 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.454947 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.476167 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.476117 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl"] Jul 20 13:02:07.476247 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.476230 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.478716 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.478699 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 13:02:07.478716 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.478720 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 13:02:07.478860 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.478745 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hjf8r\"" Jul 20 13:02:07.498310 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498286 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/031c9773-aee4-42f0-ab1e-68925940613b-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.498402 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50bbd98b-5610-4ad4-a413-3e4eec07581e-tmp\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.498402 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-default-certificate\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.498402 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498383 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.498510 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498413 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50bbd98b-5610-4ad4-a413-3e4eec07581e-service-ca-bundle\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.498510 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85eff64d-ea97-41ab-a625-39ae59200b81-config\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.498584 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-ca-trust-extracted\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.498584 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.498584 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rq9\" (UniqueName: \"kubernetes.io/projected/9f66766f-2ab9-433d-aee8-fda507d5c768-kube-api-access-76rq9\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.498700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj6hl\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-kube-api-access-hj6hl\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.498700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bp9\" (UniqueName: \"kubernetes.io/projected/85eff64d-ea97-41ab-a625-39ae59200b81-kube-api-access-b9bp9\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.498700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498643 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/031c9773-aee4-42f0-ab1e-68925940613b-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.498700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50bbd98b-5610-4ad4-a413-3e4eec07581e-tmp\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.498921 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498709 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/50bbd98b-5610-4ad4-a413-3e4eec07581e-snapshots\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.498921 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498752 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9f66766f-2ab9-433d-aee8-fda507d5c768-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.498921 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-certificates\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.498921 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74nk6\" (UniqueName: \"kubernetes.io/projected/50bbd98b-5610-4ad4-a413-3e4eec07581e-kube-api-access-74nk6\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.498921 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-bound-sa-token\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.499129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498973 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50bbd98b-5610-4ad4-a413-3e4eec07581e-service-ca-bundle\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.499129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.499129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.498978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85eff64d-ea97-41ab-a625-39ae59200b81-config\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.499129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2pq4\" (UniqueName: \"kubernetes.io/projected/031c9773-aee4-42f0-ab1e-68925940613b-kube-api-access-c2pq4\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.499129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-image-registry-private-configuration\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.499129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-stats-auth\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499142 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499172 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-nhh9k"] Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50bbd98b-5610-4ad4-a413-3e4eec07581e-serving-cert\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/50bbd98b-5610-4ad4-a413-3e4eec07581e-snapshots\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499203 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5655955bbb-7fznj"] Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45mv\" (UniqueName: \"kubernetes.io/projected/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-kube-api-access-c45mv\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vlw8\" (UniqueName: \"kubernetes.io/projected/ddaf4c2e-4bab-4ce0-8185-f95df74b5369-kube-api-access-6vlw8\") pod \"volume-data-source-validator-78cf4c6f6-s4j7s\" (UID: \"ddaf4c2e-4bab-4ce0-8185-f95df74b5369\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499286 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499256 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d66f77fc9-zckp5"] Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499313 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-installation-pull-secrets\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499336 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-69txc"] Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499339 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-trusted-ca\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499367 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50bbd98b-5610-4ad4-a413-3e4eec07581e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.499392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.499395 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85eff64d-ea97-41ab-a625-39ae59200b81-serving-cert\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.500557 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.500491 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50bbd98b-5610-4ad4-a413-3e4eec07581e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.502152 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.502130 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-csd2n\"" Jul 20 13:02:07.502470 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.502449 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.503658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.502872 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.504982 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.504960 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85eff64d-ea97-41ab-a625-39ae59200b81-serving-cert\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.505089 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.505020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50bbd98b-5610-4ad4-a413-3e4eec07581e-serving-cert\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.507945 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.507921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74nk6\" (UniqueName: \"kubernetes.io/projected/50bbd98b-5610-4ad4-a413-3e4eec07581e-kube-api-access-74nk6\") pod \"insights-operator-555d9d7885-jnvhs\" (UID: \"50bbd98b-5610-4ad4-a413-3e4eec07581e\") " pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.508035 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.507979 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bp9\" (UniqueName: \"kubernetes.io/projected/85eff64d-ea97-41ab-a625-39ae59200b81-kube-api-access-b9bp9\") pod \"service-ca-operator-5bcbc94c78-727ql\" (UID: \"85eff64d-ea97-41ab-a625-39ae59200b81\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.526771 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.526749 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl"] Jul 20 13:02:07.526864 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.526776 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl"] Jul 20 13:02:07.526864 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.526792 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wgngz"] Jul 20 13:02:07.526864 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.526808 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-69txc"] Jul 20 13:02:07.527042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.526872 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.529742 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.529725 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 13:02:07.529967 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.529941 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 13:02:07.529967 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.529948 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6btvh\"" Jul 20 13:02:07.530129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.529948 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 13:02:07.600154 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rq9\" (UniqueName: \"kubernetes.io/projected/9f66766f-2ab9-433d-aee8-fda507d5c768-kube-api-access-76rq9\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.600296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:07.600296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj6hl\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-kube-api-access-hj6hl\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.600296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600211 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72577\" (UniqueName: \"kubernetes.io/projected/dd56af17-9640-45a3-94c0-b43da624b5cd-kube-api-access-72577\") pod \"network-check-source-7bc5b6bf85-bfhzl\" (UID: \"dd56af17-9640-45a3-94c0-b43da624b5cd\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" Jul 20 13:02:07.600296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/031c9773-aee4-42f0-ab1e-68925940613b-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.600296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9f66766f-2ab9-433d-aee8-fda507d5c768-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.600525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-certificates\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.600525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600391 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.600525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600422 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-bound-sa-token\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.600525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600458 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.600525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2pq4\" (UniqueName: \"kubernetes.io/projected/031c9773-aee4-42f0-ab1e-68925940613b-kube-api-access-c2pq4\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.600525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-image-registry-private-configuration\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.600818 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-stats-auth\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.600818 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.600818 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.600622 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 13:02:07.600971 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600933 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/031c9773-aee4-42f0-ab1e-68925940613b-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.601013 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.600981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-certificates\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.601013 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.600997 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:08.100554409 +0000 UTC m=+38.652447118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : configmap references non-existent config key: service-ca.crt Jul 20 13:02:07.601094 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.601042 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:08.101027429 +0000 UTC m=+38.652920113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : secret "router-metrics-certs-default" not found Jul 20 13:02:07.601094 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-trusted-ca\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.601167 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601095 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9f66766f-2ab9-433d-aee8-fda507d5c768-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.601167 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45mv\" (UniqueName: \"kubernetes.io/projected/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-kube-api-access-c45mv\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.601167 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vlw8\" (UniqueName: \"kubernetes.io/projected/ddaf4c2e-4bab-4ce0-8185-f95df74b5369-kube-api-access-6vlw8\") pod \"volume-data-source-validator-78cf4c6f6-s4j7s\" (UID: \"ddaf4c2e-4bab-4ce0-8185-f95df74b5369\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" Jul 20 13:02:07.601276 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601242 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f037b6c-1ab5-453f-a697-9366f4eaebb2-config-volume\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.601322 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601270 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f037b6c-1ab5-453f-a697-9366f4eaebb2-tmp-dir\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.601322 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-installation-pull-secrets\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.601392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601321 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss29q\" (UniqueName: \"kubernetes.io/projected/8f037b6c-1ab5-453f-a697-9366f4eaebb2-kube-api-access-ss29q\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.601392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-trusted-ca\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.601392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601382 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7w7q\" (UniqueName: \"kubernetes.io/projected/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-kube-api-access-l7w7q\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.601496 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/031c9773-aee4-42f0-ab1e-68925940613b-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.601496 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-config\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.601585 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-default-certificate\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.601585 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601525 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.601585 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkc5n\" (UniqueName: \"kubernetes.io/projected/3b2b26a9-97cf-4c55-a817-24337c80e66b-kube-api-access-vkc5n\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:07.601585 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601580 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-serving-cert\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.601779 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601625 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-ca-trust-extracted\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.601779 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.601658 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.601779 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.601701 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 13:02:07.601779 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.601719 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d66f77fc9-zckp5: secret "image-registry-tls" not found Jul 20 13:02:07.601779 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.601768 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:07.602040 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.601788 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls podName:5107022c-b96e-4ed0-8a15-7cfb1c92ff74 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:08.101770197 +0000 UTC m=+38.653662878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls") pod "image-registry-6d66f77fc9-zckp5" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74") : secret "image-registry-tls" not found Jul 20 13:02:07.602134 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.601839 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls podName:9f66766f-2ab9-433d-aee8-fda507d5c768 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:08.101822381 +0000 UTC m=+38.653715075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mwnjl" (UID: "9f66766f-2ab9-433d-aee8-fda507d5c768") : secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:07.602205 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.602098 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-ca-trust-extracted\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.602366 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.602345 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-trusted-ca\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.603468 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.603446 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-image-registry-private-configuration\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.603653 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.603626 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-stats-auth\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.603791 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.603771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/031c9773-aee4-42f0-ab1e-68925940613b-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.604200 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.604177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-installation-pull-secrets\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.605132 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.605118 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" Jul 20 13:02:07.610330 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.610277 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rq9\" (UniqueName: \"kubernetes.io/projected/9f66766f-2ab9-433d-aee8-fda507d5c768-kube-api-access-76rq9\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:07.610330 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.610321 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-bound-sa-token\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.610557 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.610533 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45mv\" (UniqueName: \"kubernetes.io/projected/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-kube-api-access-c45mv\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.610662 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.610539 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vlw8\" (UniqueName: \"kubernetes.io/projected/ddaf4c2e-4bab-4ce0-8185-f95df74b5369-kube-api-access-6vlw8\") pod \"volume-data-source-validator-78cf4c6f6-s4j7s\" (UID: \"ddaf4c2e-4bab-4ce0-8185-f95df74b5369\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" Jul 20 13:02:07.610927 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.610905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2pq4\" (UniqueName: \"kubernetes.io/projected/031c9773-aee4-42f0-ab1e-68925940613b-kube-api-access-c2pq4\") pod \"kube-storage-version-migrator-operator-7658d9b749-cfqrm\" (UID: \"031c9773-aee4-42f0-ab1e-68925940613b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.611287 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.611269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj6hl\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-kube-api-access-hj6hl\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:07.612758 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.612742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-default-certificate\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:07.619376 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.619358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" Jul 20 13:02:07.640072 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.640058 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" Jul 20 13:02:07.659769 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.659733 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" Jul 20 13:02:07.702712 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.702678 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72577\" (UniqueName: \"kubernetes.io/projected/dd56af17-9640-45a3-94c0-b43da624b5cd-kube-api-access-72577\") pod \"network-check-source-7bc5b6bf85-bfhzl\" (UID: \"dd56af17-9640-45a3-94c0-b43da624b5cd\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" Jul 20 13:02:07.702917 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.702738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.702917 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.702798 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-trusted-ca\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.703064 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.702930 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 13:02:07.703064 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.703017 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls podName:8f037b6c-1ab5-453f-a697-9366f4eaebb2 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:08.202994673 +0000 UTC m=+38.754887370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls") pod "dns-default-wgngz" (UID: "8f037b6c-1ab5-453f-a697-9366f4eaebb2") : secret "dns-default-metrics-tls" not found Jul 20 13:02:07.703064 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.702934 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f037b6c-1ab5-453f-a697-9366f4eaebb2-config-volume\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.703064 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f037b6c-1ab5-453f-a697-9366f4eaebb2-tmp-dir\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.703288 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvwqc\" (UniqueName: \"kubernetes.io/projected/f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c-kube-api-access-pvwqc\") pod \"ingress-canary-69txc\" (UID: \"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c\") " pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.703288 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss29q\" (UniqueName: \"kubernetes.io/projected/8f037b6c-1ab5-453f-a697-9366f4eaebb2-kube-api-access-ss29q\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.703288 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7w7q\" (UniqueName: \"kubernetes.io/projected/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-kube-api-access-l7w7q\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.703288 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-config\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.703288 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkc5n\" (UniqueName: \"kubernetes.io/projected/3b2b26a9-97cf-4c55-a817-24337c80e66b-kube-api-access-vkc5n\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:07.703288 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703280 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-serving-cert\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.703559 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703313 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c-cert\") pod \"ingress-canary-69txc\" (UID: \"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c\") " pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.703559 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:07.703559 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.703475 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 13:02:07.703559 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:07.703512 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls podName:3b2b26a9-97cf-4c55-a817-24337c80e66b nodeName:}" failed. No retries permitted until 2026-07-20 13:02:08.203500791 +0000 UTC m=+38.755393468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4v4zm" (UID: "3b2b26a9-97cf-4c55-a817-24337c80e66b") : secret "samples-operator-tls" not found Jul 20 13:02:07.703559 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703527 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f037b6c-1ab5-453f-a697-9366f4eaebb2-config-volume\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.703802 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703668 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f037b6c-1ab5-453f-a697-9366f4eaebb2-tmp-dir\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.703992 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.703955 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-trusted-ca\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.704315 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.704295 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-config\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.706465 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.706431 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-serving-cert\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.722600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.720711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72577\" (UniqueName: \"kubernetes.io/projected/dd56af17-9640-45a3-94c0-b43da624b5cd-kube-api-access-72577\") pod \"network-check-source-7bc5b6bf85-bfhzl\" (UID: \"dd56af17-9640-45a3-94c0-b43da624b5cd\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" Jul 20 13:02:07.722600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.721123 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkc5n\" (UniqueName: \"kubernetes.io/projected/3b2b26a9-97cf-4c55-a817-24337c80e66b-kube-api-access-vkc5n\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:07.722600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.721340 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss29q\" (UniqueName: \"kubernetes.io/projected/8f037b6c-1ab5-453f-a697-9366f4eaebb2-kube-api-access-ss29q\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:07.722600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.722331 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7w7q\" (UniqueName: \"kubernetes.io/projected/212913d4-c0a2-4d4c-a2fc-94c38b2be88f-kube-api-access-l7w7q\") pod \"console-operator-565b7b569d-nhh9k\" (UID: \"212913d4-c0a2-4d4c-a2fc-94c38b2be88f\") " pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.740863 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.740565 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:07.805237 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.804675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c-cert\") pod \"ingress-canary-69txc\" (UID: \"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c\") " pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.805237 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.804809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvwqc\" (UniqueName: \"kubernetes.io/projected/f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c-kube-api-access-pvwqc\") pod \"ingress-canary-69txc\" (UID: \"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c\") " pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.808412 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.808383 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c-cert\") pod \"ingress-canary-69txc\" (UID: \"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c\") " pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.815452 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.815392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvwqc\" (UniqueName: \"kubernetes.io/projected/f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c-kube-api-access-pvwqc\") pod \"ingress-canary-69txc\" (UID: \"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c\") " pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.819163 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.819133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" Jul 20 13:02:07.834345 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.834282 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-69txc" Jul 20 13:02:07.846668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.846532 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql"] Jul 20 13:02:07.850666 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:07.850636 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85eff64d_ea97_41ab_a625_39ae59200b81.slice/crio-f1e62ac0e0f37e438ca73b7488ea8815eff7791a423647a876abc435d53f79c7 WatchSource:0}: Error finding container f1e62ac0e0f37e438ca73b7488ea8815eff7791a423647a876abc435d53f79c7: Status 404 returned error can't find the container with id f1e62ac0e0f37e438ca73b7488ea8815eff7791a423647a876abc435d53f79c7 Jul 20 13:02:07.851686 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.851660 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm"] Jul 20 13:02:07.854753 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.854728 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s"] Jul 20 13:02:07.856343 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:07.856309 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031c9773_aee4_42f0_ab1e_68925940613b.slice/crio-33b8148d5f9e9ce289410d78b763832f50f06a5570ca20b85fddc60e6782b273 WatchSource:0}: Error finding container 33b8148d5f9e9ce289410d78b763832f50f06a5570ca20b85fddc60e6782b273: Status 404 returned error can't find the container with id 33b8148d5f9e9ce289410d78b763832f50f06a5570ca20b85fddc60e6782b273 Jul 20 13:02:07.858171 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:07.858146 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddaf4c2e_4bab_4ce0_8185_f95df74b5369.slice/crio-336a2d6c6788b09ae3e912e9d76c2565548daf6c3c6870981ae658f9e4d72252 WatchSource:0}: Error finding container 336a2d6c6788b09ae3e912e9d76c2565548daf6c3c6870981ae658f9e4d72252: Status 404 returned error can't find the container with id 336a2d6c6788b09ae3e912e9d76c2565548daf6c3c6870981ae658f9e4d72252 Jul 20 13:02:07.858801 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.858751 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jnvhs"] Jul 20 13:02:07.860063 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:07.859983 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50bbd98b_5610_4ad4_a413_3e4eec07581e.slice/crio-617e113223bd7a9d9d70af7738067b0b1e422c147dd38234336a8ef356eab7d0 WatchSource:0}: Error finding container 617e113223bd7a9d9d70af7738067b0b1e422c147dd38234336a8ef356eab7d0: Status 404 returned error can't find the container with id 617e113223bd7a9d9d70af7738067b0b1e422c147dd38234336a8ef356eab7d0 Jul 20 13:02:07.926124 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:07.925845 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-nhh9k"] Jul 20 13:02:07.932597 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:07.932103 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212913d4_c0a2_4d4c_a2fc_94c38b2be88f.slice/crio-69424979ef984fb90e1bf5f261f545c2771aad9c15f5e30536f49c510b8a4c94 WatchSource:0}: Error finding container 69424979ef984fb90e1bf5f261f545c2771aad9c15f5e30536f49c510b8a4c94: Status 404 returned error can't find the container with id 69424979ef984fb90e1bf5f261f545c2771aad9c15f5e30536f49c510b8a4c94 Jul 20 13:02:08.007123 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.007056 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl"] Jul 20 13:02:08.009924 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:08.009900 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd56af17_9640_45a3_94c0_b43da624b5cd.slice/crio-2dc650fefc2f96ed6a18cfae500dfc527ab95cfaee54ef1e525390d8ad41dad5 WatchSource:0}: Error finding container 2dc650fefc2f96ed6a18cfae500dfc527ab95cfaee54ef1e525390d8ad41dad5: Status 404 returned error can't find the container with id 2dc650fefc2f96ed6a18cfae500dfc527ab95cfaee54ef1e525390d8ad41dad5 Jul 20 13:02:08.021329 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.021305 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-69txc"] Jul 20 13:02:08.023849 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:08.023828 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e9167d_2e5c_45e8_ac2d_c8f85981cd8c.slice/crio-89f1a862cd57f9cac41fbf52445b9f33198ffa380dc20a8d5d8731ca5d22de63 WatchSource:0}: Error finding container 89f1a862cd57f9cac41fbf52445b9f33198ffa380dc20a8d5d8731ca5d22de63: Status 404 returned error can't find the container with id 89f1a862cd57f9cac41fbf52445b9f33198ffa380dc20a8d5d8731ca5d22de63 Jul 20 13:02:08.108149 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.108118 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:08.108306 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.108184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:08.108306 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.108214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:08.108306 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.108274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:08.108306 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108296 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108370 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:09.108351196 +0000 UTC m=+39.660243893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : configmap references non-existent config key: service-ca.crt Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108374 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108393 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls podName:9f66766f-2ab9-433d-aee8-fda507d5c768 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:09.108383929 +0000 UTC m=+39.660276608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mwnjl" (UID: "9f66766f-2ab9-433d-aee8-fda507d5c768") : secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108429 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:09.108415662 +0000 UTC m=+39.660308339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : secret "router-metrics-certs-default" not found Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108374 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108449 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d66f77fc9-zckp5: secret "image-registry-tls" not found Jul 20 13:02:08.108513 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.108469 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls podName:5107022c-b96e-4ed0-8a15-7cfb1c92ff74 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:09.108462453 +0000 UTC m=+39.660355129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls") pod "image-registry-6d66f77fc9-zckp5" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74") : secret "image-registry-tls" not found Jul 20 13:02:08.137400 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.137369 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" event={"ID":"dd56af17-9640-45a3-94c0-b43da624b5cd","Type":"ContainerStarted","Data":"2dc650fefc2f96ed6a18cfae500dfc527ab95cfaee54ef1e525390d8ad41dad5"} Jul 20 13:02:08.138431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.138407 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" event={"ID":"031c9773-aee4-42f0-ab1e-68925940613b","Type":"ContainerStarted","Data":"33b8148d5f9e9ce289410d78b763832f50f06a5570ca20b85fddc60e6782b273"} Jul 20 13:02:08.139505 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.139481 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" event={"ID":"85eff64d-ea97-41ab-a625-39ae59200b81","Type":"ContainerStarted","Data":"f1e62ac0e0f37e438ca73b7488ea8815eff7791a423647a876abc435d53f79c7"} Jul 20 13:02:08.140473 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.140451 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" event={"ID":"212913d4-c0a2-4d4c-a2fc-94c38b2be88f","Type":"ContainerStarted","Data":"69424979ef984fb90e1bf5f261f545c2771aad9c15f5e30536f49c510b8a4c94"} Jul 20 13:02:08.141491 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.141473 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-69txc" event={"ID":"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c","Type":"ContainerStarted","Data":"89f1a862cd57f9cac41fbf52445b9f33198ffa380dc20a8d5d8731ca5d22de63"} Jul 20 13:02:08.142446 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.142429 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" event={"ID":"ddaf4c2e-4bab-4ce0-8185-f95df74b5369","Type":"ContainerStarted","Data":"336a2d6c6788b09ae3e912e9d76c2565548daf6c3c6870981ae658f9e4d72252"} Jul 20 13:02:08.143349 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.143332 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" event={"ID":"50bbd98b-5610-4ad4-a413-3e4eec07581e","Type":"ContainerStarted","Data":"617e113223bd7a9d9d70af7738067b0b1e422c147dd38234336a8ef356eab7d0"} Jul 20 13:02:08.209386 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.209356 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:08.209495 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.209400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:08.209546 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.209512 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 13:02:08.209581 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.209564 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 13:02:08.209613 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.209571 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls podName:3b2b26a9-97cf-4c55-a817-24337c80e66b nodeName:}" failed. No retries permitted until 2026-07-20 13:02:09.209555408 +0000 UTC m=+39.761448090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4v4zm" (UID: "3b2b26a9-97cf-4c55-a817-24337c80e66b") : secret "samples-operator-tls" not found Jul 20 13:02:08.209660 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:08.209622 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls podName:8f037b6c-1ab5-453f-a697-9366f4eaebb2 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:09.209609664 +0000 UTC m=+39.761502340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls") pod "dns-default-wgngz" (UID: "8f037b6c-1ab5-453f-a697-9366f4eaebb2") : secret "dns-default-metrics-tls" not found Jul 20 13:02:08.931526 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.929709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:08.931526 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.930481 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:08.931526 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.930935 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:08.934061 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.933516 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 13:02:08.934061 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.933867 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 13:02:08.936026 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.935799 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m99kw\"" Jul 20 13:02:08.936026 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:08.935919 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dbwl5\"" Jul 20 13:02:09.120718 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:09.120673 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:09.120946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:09.120778 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:09.120946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:09.120864 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:09.120946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:09.120912 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:09.121131 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.121091 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 13:02:09.121798 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.121773 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 13:02:09.121798 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.121800 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d66f77fc9-zckp5: secret "image-registry-tls" not found Jul 20 13:02:09.121982 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.121862 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls podName:5107022c-b96e-4ed0-8a15-7cfb1c92ff74 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:11.121840973 +0000 UTC m=+41.673733656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls") pod "image-registry-6d66f77fc9-zckp5" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74") : secret "image-registry-tls" not found Jul 20 13:02:09.122308 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.122280 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:09.122400 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.122338 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls podName:9f66766f-2ab9-433d-aee8-fda507d5c768 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:11.122322325 +0000 UTC m=+41.674215006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mwnjl" (UID: "9f66766f-2ab9-433d-aee8-fda507d5c768") : secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:09.122466 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.122419 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:11.122407019 +0000 UTC m=+41.674299702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : configmap references non-existent config key: service-ca.crt Jul 20 13:02:09.122827 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.122798 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:11.122778416 +0000 UTC m=+41.674671098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : secret "router-metrics-certs-default" not found Jul 20 13:02:09.223047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:09.222233 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:09.223047 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:09.222318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:09.223047 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.222507 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 13:02:09.223047 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.222568 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls podName:8f037b6c-1ab5-453f-a697-9366f4eaebb2 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:11.222549818 +0000 UTC m=+41.774442499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls") pod "dns-default-wgngz" (UID: "8f037b6c-1ab5-453f-a697-9366f4eaebb2") : secret "dns-default-metrics-tls" not found Jul 20 13:02:09.223047 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.222964 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 13:02:09.223047 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:09.223015 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls podName:3b2b26a9-97cf-4c55-a817-24337c80e66b nodeName:}" failed. No retries permitted until 2026-07-20 13:02:11.223000161 +0000 UTC m=+41.774892843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4v4zm" (UID: "3b2b26a9-97cf-4c55-a817-24337c80e66b") : secret "samples-operator-tls" not found Jul 20 13:02:11.144086 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:11.143992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144165 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:11.144189 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:11.144218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144237 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls podName:9f66766f-2ab9-433d-aee8-fda507d5c768 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:15.14421598 +0000 UTC m=+45.696108657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mwnjl" (UID: "9f66766f-2ab9-433d-aee8-fda507d5c768") : secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144294 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144308 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:15.144295371 +0000 UTC m=+45.696188052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : configmap references non-existent config key: service-ca.crt Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144341 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:15.144326511 +0000 UTC m=+45.696219216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : secret "router-metrics-certs-default" not found Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:11.144396 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144569 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144588 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d66f77fc9-zckp5: secret "image-registry-tls" not found Jul 20 13:02:11.144668 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.144641 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls podName:5107022c-b96e-4ed0-8a15-7cfb1c92ff74 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:15.144626757 +0000 UTC m=+45.696519436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls") pod "image-registry-6d66f77fc9-zckp5" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74") : secret "image-registry-tls" not found Jul 20 13:02:11.245299 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:11.245265 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:11.245423 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:11.245388 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:11.245457 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.245414 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 13:02:11.245499 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.245468 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls podName:8f037b6c-1ab5-453f-a697-9366f4eaebb2 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:15.245453513 +0000 UTC m=+45.797346189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls") pod "dns-default-wgngz" (UID: "8f037b6c-1ab5-453f-a697-9366f4eaebb2") : secret "dns-default-metrics-tls" not found Jul 20 13:02:11.245499 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.245473 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 13:02:11.245575 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:11.245509 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls podName:3b2b26a9-97cf-4c55-a817-24337c80e66b nodeName:}" failed. No retries permitted until 2026-07-20 13:02:15.245498317 +0000 UTC m=+45.797390994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4v4zm" (UID: "3b2b26a9-97cf-4c55-a817-24337c80e66b") : secret "samples-operator-tls" not found Jul 20 13:02:15.185705 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:15.185666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:15.185761 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185791 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185816 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d66f77fc9-zckp5: secret "image-registry-tls" not found Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:15.185848 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185897 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls podName:5107022c-b96e-4ed0-8a15-7cfb1c92ff74 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:23.185858684 +0000 UTC m=+53.737751364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls") pod "image-registry-6d66f77fc9-zckp5" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74") : secret "image-registry-tls" not found Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185938 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:15.185947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185971 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:23.185955704 +0000 UTC m=+53.737848383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : configmap references non-existent config key: service-ca.crt Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185994 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls podName:9f66766f-2ab9-433d-aee8-fda507d5c768 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:23.185985025 +0000 UTC m=+53.737877701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mwnjl" (UID: "9f66766f-2ab9-433d-aee8-fda507d5c768") : secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.185999 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 13:02:15.186437 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.186040 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:23.186028902 +0000 UTC m=+53.737921582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : secret "router-metrics-certs-default" not found Jul 20 13:02:15.287039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:15.286756 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:15.287039 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.287005 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 13:02:15.287039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:15.287023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:15.287242 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.287076 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls podName:3b2b26a9-97cf-4c55-a817-24337c80e66b nodeName:}" failed. No retries permitted until 2026-07-20 13:02:23.287055633 +0000 UTC m=+53.838948317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4v4zm" (UID: "3b2b26a9-97cf-4c55-a817-24337c80e66b") : secret "samples-operator-tls" not found Jul 20 13:02:15.287242 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.287131 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 13:02:15.287242 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:15.287187 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls podName:8f037b6c-1ab5-453f-a697-9366f4eaebb2 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:23.287172503 +0000 UTC m=+53.839065203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls") pod "dns-default-wgngz" (UID: "8f037b6c-1ab5-453f-a697-9366f4eaebb2") : secret "dns-default-metrics-tls" not found Jul 20 13:02:16.171730 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.171696 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" event={"ID":"ddaf4c2e-4bab-4ce0-8185-f95df74b5369","Type":"ContainerStarted","Data":"98d610819f81abca173b82ce6d1df8a09677c1f97e0c9b5cfeca51de63e9b7aa"} Jul 20 13:02:16.173924 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.173897 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" event={"ID":"50bbd98b-5610-4ad4-a413-3e4eec07581e","Type":"ContainerStarted","Data":"eb13549444b17136f7df7887922194162932dcf56715d251ad03ae30eab8d6b7"} Jul 20 13:02:16.175574 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.175549 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" event={"ID":"dd56af17-9640-45a3-94c0-b43da624b5cd","Type":"ContainerStarted","Data":"55dce3366ae099cd7a506f7802f3709153ea8eb634d3b1f50aa761f0ffc1b028"} Jul 20 13:02:16.177482 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.177462 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" event={"ID":"031c9773-aee4-42f0-ab1e-68925940613b","Type":"ContainerStarted","Data":"ec6f0a4495dee0b3d6a07deb1c3fbcaae8659fd34b87ea245ddfd2c727188a97"} Jul 20 13:02:16.179065 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.179014 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" event={"ID":"85eff64d-ea97-41ab-a625-39ae59200b81","Type":"ContainerStarted","Data":"8dc880f51fa611a37f37a0070bc4b3e5da62b4a2a3ab825767ea2858dd7941bc"} Jul 20 13:02:16.180684 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.180668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/0.log" Jul 20 13:02:16.180786 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.180702 2564 generic.go:356] "Generic (PLEG): container finished" podID="212913d4-c0a2-4d4c-a2fc-94c38b2be88f" containerID="340104e9941923dcaccc687ab38f3fe69bb831d156701c7d8a86b09bd37531db" exitCode=255 Jul 20 13:02:16.180786 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.180772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" event={"ID":"212913d4-c0a2-4d4c-a2fc-94c38b2be88f","Type":"ContainerDied","Data":"340104e9941923dcaccc687ab38f3fe69bb831d156701c7d8a86b09bd37531db"} Jul 20 13:02:16.181086 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.181041 2564 scope.go:117] "RemoveContainer" containerID="340104e9941923dcaccc687ab38f3fe69bb831d156701c7d8a86b09bd37531db" Jul 20 13:02:16.182571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.182550 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-69txc" event={"ID":"f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c","Type":"ContainerStarted","Data":"928ee464335f0275c995a05ff2ddb0ff06f11b6dc04b9cd6c812bdc847ba5a65"} Jul 20 13:02:16.189455 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.189412 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-s4j7s" podStartSLOduration=21.121121633 podStartE2EDuration="23.189396926s" totalImagesPullingTime="2.068275293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:16.189147522 +0000 UTC m=+46.741040245" watchObservedRunningTime="2026-07-20 13:02:16.189396926 +0000 UTC m=+46.741289623" Jul 20 13:02:16.225123 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.224861 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-bfhzl" podStartSLOduration=15.990172258 podStartE2EDuration="23.224842095s" totalImagesPullingTime="7.234669837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:16.224221698 +0000 UTC m=+46.776114410" watchObservedRunningTime="2026-07-20 13:02:16.224842095 +0000 UTC m=+46.776734796" Jul 20 13:02:16.242603 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.242355 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" podStartSLOduration=15.858434487 podStartE2EDuration="23.242336806s" totalImagesPullingTime="7.383902319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:16.241185306 +0000 UTC m=+46.793078006" watchObservedRunningTime="2026-07-20 13:02:16.242336806 +0000 UTC m=+46.794229487" Jul 20 13:02:16.260458 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.260411 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-69txc" podStartSLOduration=2.039712602 podStartE2EDuration="9.260391626s" totalImagesPullingTime="7.220679024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:16.258905314 +0000 UTC m=+46.810798034" watchObservedRunningTime="2026-07-20 13:02:16.260391626 +0000 UTC m=+46.812284326" Jul 20 13:02:16.310770 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.310701 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" podStartSLOduration=15.687676725 podStartE2EDuration="23.310680487s" totalImagesPullingTime="7.623003762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:16.291279242 +0000 UTC m=+46.843171967" watchObservedRunningTime="2026-07-20 13:02:16.310680487 +0000 UTC m=+46.862573187" Jul 20 13:02:16.311488 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.311451 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" podStartSLOduration=15.923167248 podStartE2EDuration="23.311424352s" totalImagesPullingTime="7.388257104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:16.309087807 +0000 UTC m=+46.860980506" watchObservedRunningTime="2026-07-20 13:02:16.311424352 +0000 UTC m=+46.863317052" Jul 20 13:02:16.585318 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.585275 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-njlnn"] Jul 20 13:02:16.600324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.600289 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.603554 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.603524 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-njlnn"] Jul 20 13:02:16.604869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.604656 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 13:02:16.604869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.604713 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 13:02:16.604869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.604728 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rdx8c\"" Jul 20 13:02:16.701426 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.701245 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-data-volume\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.701592 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.701474 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4mm2\" (UniqueName: \"kubernetes.io/projected/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-api-access-h4mm2\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.701592 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.701512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.701592 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.701528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.701723 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.701629 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-crio-socket\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.802489 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.802455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.802489 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.802490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.802721 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.802577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-crio-socket\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.802721 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:16.802580 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 13:02:16.802721 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:16.802643 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 13:02:16.802721 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:16.802658 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:17.302638456 +0000 UTC m=+47.854531139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : configmap "kube-rbac-proxy" not found Jul 20 13:02:16.802721 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:16.802679 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:17.302668919 +0000 UTC m=+47.854561595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : secret "insights-runtime-extractor-tls" not found Jul 20 13:02:16.802986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.802728 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-data-volume\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.802986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.802788 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4mm2\" (UniqueName: \"kubernetes.io/projected/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-api-access-h4mm2\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.802986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.802802 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-crio-socket\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.803329 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.803311 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-data-volume\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:16.813598 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:16.813571 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4mm2\" (UniqueName: \"kubernetes.io/projected/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-api-access-h4mm2\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:17.188403 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.188374 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:02:17.188855 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.188836 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/0.log" Jul 20 13:02:17.188962 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.188895 2564 generic.go:356] "Generic (PLEG): container finished" podID="212913d4-c0a2-4d4c-a2fc-94c38b2be88f" containerID="0566f428e2fc8fda3f270076c68d1995326b5a8810a75b67089b5a3c47e8128f" exitCode=255 Jul 20 13:02:17.189015 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.188975 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" event={"ID":"212913d4-c0a2-4d4c-a2fc-94c38b2be88f","Type":"ContainerDied","Data":"0566f428e2fc8fda3f270076c68d1995326b5a8810a75b67089b5a3c47e8128f"} Jul 20 13:02:17.189066 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.189037 2564 scope.go:117] "RemoveContainer" containerID="340104e9941923dcaccc687ab38f3fe69bb831d156701c7d8a86b09bd37531db" Jul 20 13:02:17.189325 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.189288 2564 scope.go:117] "RemoveContainer" containerID="0566f428e2fc8fda3f270076c68d1995326b5a8810a75b67089b5a3c47e8128f" Jul 20 13:02:17.189493 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:17.189469 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-nhh9k_openshift-console-operator(212913d4-c0a2-4d4c-a2fc-94c38b2be88f)\"" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" podUID="212913d4-c0a2-4d4c-a2fc-94c38b2be88f" Jul 20 13:02:17.308123 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.307589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:17.308123 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:17.307665 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 13:02:17.308123 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.307666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:17.308123 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:17.307718 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:18.307700934 +0000 UTC m=+48.859593618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : configmap "kube-rbac-proxy" not found Jul 20 13:02:17.308123 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:17.307763 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 13:02:17.308123 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:17.307807 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:18.307797473 +0000 UTC m=+48.859690164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : secret "insights-runtime-extractor-tls" not found Jul 20 13:02:17.741799 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.741769 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:17.741993 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.741819 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:17.984240 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:17.984213 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6klgv_65a6fc4f-a16e-45bf-80c5-bb971c414db4/dns-node-resolver/0.log" Jul 20 13:02:18.193429 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:18.193406 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:02:18.193932 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:18.193712 2564 scope.go:117] "RemoveContainer" containerID="0566f428e2fc8fda3f270076c68d1995326b5a8810a75b67089b5a3c47e8128f" Jul 20 13:02:18.193932 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:18.193851 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-nhh9k_openshift-console-operator(212913d4-c0a2-4d4c-a2fc-94c38b2be88f)\"" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" podUID="212913d4-c0a2-4d4c-a2fc-94c38b2be88f" Jul 20 13:02:18.317196 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:18.317163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:18.317355 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:18.317232 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 13:02:18.317355 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:18.317282 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:20.317264723 +0000 UTC m=+50.869157415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : configmap "kube-rbac-proxy" not found Jul 20 13:02:18.317355 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:18.317297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:18.317489 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:18.317475 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 13:02:18.317535 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:18.317524 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:20.317510584 +0000 UTC m=+50.869403267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : secret "insights-runtime-extractor-tls" not found Jul 20 13:02:18.984856 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:18.984829 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c6kfj_0a87dc87-258f-47ea-b5a4-1e5656f6a80c/node-ca/0.log" Jul 20 13:02:19.197319 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:19.197295 2564 scope.go:117] "RemoveContainer" containerID="0566f428e2fc8fda3f270076c68d1995326b5a8810a75b67089b5a3c47e8128f" Jul 20 13:02:19.197667 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:19.197443 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-nhh9k_openshift-console-operator(212913d4-c0a2-4d4c-a2fc-94c38b2be88f)\"" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" podUID="212913d4-c0a2-4d4c-a2fc-94c38b2be88f" Jul 20 13:02:19.787775 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:19.787744 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-69txc_f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c/serve-healthcheck-canary/0.log" Jul 20 13:02:20.335563 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:20.335513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:20.335563 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:20.335564 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:20.336085 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:20.335639 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 13:02:20.336085 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:20.335706 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:24.335689377 +0000 UTC m=+54.887582058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : configmap "kube-rbac-proxy" not found Jul 20 13:02:20.336085 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:20.335769 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 13:02:20.336085 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:20.335847 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:24.335828776 +0000 UTC m=+54.887721473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : secret "insights-runtime-extractor-tls" not found Jul 20 13:02:23.260969 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:23.260930 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:23.260987 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261077 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261094 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d66f77fc9-zckp5: secret "image-registry-tls" not found Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261080 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261142 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls podName:5107022c-b96e-4ed0-8a15-7cfb1c92ff74 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:39.261127067 +0000 UTC m=+69.813019744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls") pod "image-registry-6d66f77fc9-zckp5" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74") : secret "image-registry-tls" not found Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261165 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls podName:9f66766f-2ab9-433d-aee8-fda507d5c768 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:39.261154994 +0000 UTC m=+69.813047674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mwnjl" (UID: "9f66766f-2ab9-433d-aee8-fda507d5c768") : secret "cluster-monitoring-operator-tls" not found Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:23.261179 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:23.261211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261292 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:39.261278198 +0000 UTC m=+69.813170889 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : configmap references non-existent config key: service-ca.crt Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261318 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 13:02:23.261344 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.261344 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs podName:2116eb31-99a1-4ab5-848e-ebf5e6c4f938 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:39.261335882 +0000 UTC m=+69.813228560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs") pod "router-default-5655955bbb-7fznj" (UID: "2116eb31-99a1-4ab5-848e-ebf5e6c4f938") : secret "router-metrics-certs-default" not found Jul 20 13:02:23.361707 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:23.361670 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:23.361863 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:23.361783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:23.361863 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.361809 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 13:02:23.361954 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.361894 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 13:02:23.361954 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.361897 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls podName:8f037b6c-1ab5-453f-a697-9366f4eaebb2 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:39.361865473 +0000 UTC m=+69.913758150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls") pod "dns-default-wgngz" (UID: "8f037b6c-1ab5-453f-a697-9366f4eaebb2") : secret "dns-default-metrics-tls" not found Jul 20 13:02:23.361954 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:23.361939 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls podName:3b2b26a9-97cf-4c55-a817-24337c80e66b nodeName:}" failed. No retries permitted until 2026-07-20 13:02:39.361928243 +0000 UTC m=+69.913820919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-4v4zm" (UID: "3b2b26a9-97cf-4c55-a817-24337c80e66b") : secret "samples-operator-tls" not found Jul 20 13:02:24.369463 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:24.369426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:24.369463 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:24.369463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:24.369974 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:24.369553 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 13:02:24.369974 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:24.369623 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:32.369604806 +0000 UTC m=+62.921497498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : configmap "kube-rbac-proxy" not found Jul 20 13:02:24.369974 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:24.369649 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 13:02:24.369974 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:24.369698 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:32.369686794 +0000 UTC m=+62.921579472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : secret "insights-runtime-extractor-tls" not found Jul 20 13:02:32.443287 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:32.443247 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:32.443287 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:32.443287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:32.443725 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:32.443426 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 13:02:32.443725 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:32.443481 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls podName:a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9 nodeName:}" failed. No retries permitted until 2026-07-20 13:02:48.44346565 +0000 UTC m=+78.995358348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-njlnn" (UID: "a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9") : secret "insights-runtime-extractor-tls" not found Jul 20 13:02:32.443801 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:32.443732 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:33.136467 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:33.136437 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjm7q" Jul 20 13:02:33.934270 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:33.934242 2564 scope.go:117] "RemoveContainer" containerID="0566f428e2fc8fda3f270076c68d1995326b5a8810a75b67089b5a3c47e8128f" Jul 20 13:02:34.249118 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:34.249082 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:02:34.249307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:34.249129 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" event={"ID":"212913d4-c0a2-4d4c-a2fc-94c38b2be88f","Type":"ContainerStarted","Data":"dcbfffb4dd78713f4bd9570de392d54506e9de6800e47861189aac0e43703c5a"} Jul 20 13:02:34.249462 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:34.249445 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:34.267806 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:34.267763 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" podStartSLOduration=33.955173198 podStartE2EDuration="41.26774791s" totalImagesPullingTime="7.312574712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:34.267231514 +0000 UTC m=+64.819124213" watchObservedRunningTime="2026-07-20 13:02:34.26774791 +0000 UTC m=+64.819640606" Jul 20 13:02:34.603155 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:34.603085 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-nhh9k" Jul 20 13:02:35.568517 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.568484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:35.571408 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.571391 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 13:02:35.580899 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.580854 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6bd398-f692-4fac-959b-f296c7ac6a30-metrics-certs\") pod \"network-metrics-daemon-ss8lq\" (UID: \"aa6bd398-f692-4fac-959b-f296c7ac6a30\") " pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:35.663558 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.663529 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dbwl5\"" Jul 20 13:02:35.669144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.669115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:35.669443 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.669429 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ss8lq" Jul 20 13:02:35.671731 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.671712 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 13:02:35.682058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.682019 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3791d792-594b-43fa-a1af-3ec0bf0aac2d-original-pull-secret\") pod \"global-pull-secret-syncer-xcmtb\" (UID: \"3791d792-594b-43fa-a1af-3ec0bf0aac2d\") " pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:35.770448 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.770412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:35.772976 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.772948 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bjrg\" (UniqueName: \"kubernetes.io/projected/a90281c1-8b44-462e-8f2f-7d7eaf23210f-kube-api-access-7bjrg\") pod \"network-check-target-cwqfj\" (UID: \"a90281c1-8b44-462e-8f2f-7d7eaf23210f\") " pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:35.797370 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.797345 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ss8lq"] Jul 20 13:02:35.801279 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:35.799726 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6bd398_f692_4fac_959b_f296c7ac6a30.slice/crio-7d44c75ac4fb95bac50ce59cacf36bd7ec5ae67523d55e331c7d8aa539cf6121 WatchSource:0}: Error finding container 7d44c75ac4fb95bac50ce59cacf36bd7ec5ae67523d55e331c7d8aa539cf6121: Status 404 returned error can't find the container with id 7d44c75ac4fb95bac50ce59cacf36bd7ec5ae67523d55e331c7d8aa539cf6121 Jul 20 13:02:35.949964 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.949939 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m99kw\"" Jul 20 13:02:35.958066 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.958046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:35.967525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:35.967503 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xcmtb" Jul 20 13:02:36.087246 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.087197 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cwqfj"] Jul 20 13:02:36.089715 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:36.089686 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda90281c1_8b44_462e_8f2f_7d7eaf23210f.slice/crio-58d1bbf7965522857efa869f8da8270147df0978876922c16881161a83a34003 WatchSource:0}: Error finding container 58d1bbf7965522857efa869f8da8270147df0978876922c16881161a83a34003: Status 404 returned error can't find the container with id 58d1bbf7965522857efa869f8da8270147df0978876922c16881161a83a34003 Jul 20 13:02:36.101309 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.101285 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xcmtb"] Jul 20 13:02:36.103531 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:36.103504 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3791d792_594b_43fa_a1af_3ec0bf0aac2d.slice/crio-74d93ed10ff53af0e58a6dbc203faed888793ad884b8af62a7a59cea4d19308e WatchSource:0}: Error finding container 74d93ed10ff53af0e58a6dbc203faed888793ad884b8af62a7a59cea4d19308e: Status 404 returned error can't find the container with id 74d93ed10ff53af0e58a6dbc203faed888793ad884b8af62a7a59cea4d19308e Jul 20 13:02:36.255826 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.255737 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xcmtb" event={"ID":"3791d792-594b-43fa-a1af-3ec0bf0aac2d","Type":"ContainerStarted","Data":"74d93ed10ff53af0e58a6dbc203faed888793ad884b8af62a7a59cea4d19308e"} Jul 20 13:02:36.257325 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.257296 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cwqfj" event={"ID":"a90281c1-8b44-462e-8f2f-7d7eaf23210f","Type":"ContainerStarted","Data":"95139927102fc9111c1707ea7ffac938ddb1ce257614fca3cfad4f511f4e9e13"} Jul 20 13:02:36.257470 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.257356 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cwqfj" event={"ID":"a90281c1-8b44-462e-8f2f-7d7eaf23210f","Type":"ContainerStarted","Data":"58d1bbf7965522857efa869f8da8270147df0978876922c16881161a83a34003"} Jul 20 13:02:36.257470 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.257415 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:02:36.258471 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.258450 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ss8lq" event={"ID":"aa6bd398-f692-4fac-959b-f296c7ac6a30","Type":"ContainerStarted","Data":"7d44c75ac4fb95bac50ce59cacf36bd7ec5ae67523d55e331c7d8aa539cf6121"} Jul 20 13:02:36.274726 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:36.274685 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cwqfj" podStartSLOduration=66.274672507 podStartE2EDuration="1m6.274672507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:36.273581278 +0000 UTC m=+66.825473998" watchObservedRunningTime="2026-07-20 13:02:36.274672507 +0000 UTC m=+66.826565206" Jul 20 13:02:37.264499 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:37.264460 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ss8lq" event={"ID":"aa6bd398-f692-4fac-959b-f296c7ac6a30","Type":"ContainerStarted","Data":"1e367441bc82a552a0e13025a6daaacfd16dbc2f17dca3ff1042e5a6b65d2b41"} Jul 20 13:02:37.264973 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:37.264526 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ss8lq" event={"ID":"aa6bd398-f692-4fac-959b-f296c7ac6a30","Type":"ContainerStarted","Data":"99a4e685a1b8add58070864398a9d3ac6db71614efeb5fedafc14a4a5e24a9f2"} Jul 20 13:02:37.281506 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:37.281464 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ss8lq" podStartSLOduration=66.201389559 podStartE2EDuration="1m7.281447437s" totalImagesPullingTime="1.080057878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:37.280152649 +0000 UTC m=+67.832045349" watchObservedRunningTime="2026-07-20 13:02:37.281447437 +0000 UTC m=+67.833340137" Jul 20 13:02:39.302105 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.302066 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:39.302578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.302124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:39.302578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.302200 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:39.302578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.302242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:39.302924 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.302900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-service-ca-bundle\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:39.305040 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.304996 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f66766f-2ab9-433d-aee8-fda507d5c768-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mwnjl\" (UID: \"9f66766f-2ab9-433d-aee8-fda507d5c768\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:39.305132 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.305037 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2116eb31-99a1-4ab5-848e-ebf5e6c4f938-metrics-certs\") pod \"router-default-5655955bbb-7fznj\" (UID: \"2116eb31-99a1-4ab5-848e-ebf5e6c4f938\") " pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:39.305132 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.305094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"image-registry-6d66f77fc9-zckp5\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:39.403093 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.403055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:39.403279 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.403127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:39.406166 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.406141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f037b6c-1ab5-453f-a697-9366f4eaebb2-metrics-tls\") pod \"dns-default-wgngz\" (UID: \"8f037b6c-1ab5-453f-a697-9366f4eaebb2\") " pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:39.406328 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.406263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3b2b26a9-97cf-4c55-a817-24337c80e66b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-4v4zm\" (UID: \"3b2b26a9-97cf-4c55-a817-24337c80e66b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:39.438332 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.438296 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9"] Jul 20 13:02:39.441392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.441338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.445108 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.445076 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 20 13:02:39.445254 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.445166 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 20 13:02:39.445254 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.445166 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 20 13:02:39.445608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.445541 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 20 13:02:39.445608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.445563 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 20 13:02:39.446362 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.446344 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 20 13:02:39.446597 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.446580 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 20 13:02:39.459790 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.459766 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9"] Jul 20 13:02:39.480517 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.480493 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tpf8d\"" Jul 20 13:02:39.488656 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.488625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:39.494337 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.494319 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vndj9\"" Jul 20 13:02:39.502628 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.502598 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" Jul 20 13:02:39.514993 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.514969 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nr2p9\"" Jul 20 13:02:39.523656 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.523630 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:39.562380 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.562309 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-47p9x\"" Jul 20 13:02:39.570447 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.570404 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" Jul 20 13:02:39.592390 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.592362 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hjf8r\"" Jul 20 13:02:39.594489 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.594450 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:39.604667 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.604625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.604791 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.604706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-ca\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.604791 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.604760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzw2l\" (UniqueName: \"kubernetes.io/projected/fa847739-e487-489f-8848-a97020cbc662-kube-api-access-jzw2l\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.604930 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.604907 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.605008 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.604986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-hub\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.605059 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.605040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa847739-e487-489f-8848-a97020cbc662-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.706140 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.706096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-hub\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.706314 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.706153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa847739-e487-489f-8848-a97020cbc662-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.706314 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.706203 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.706314 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.706241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-ca\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.706479 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.706391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzw2l\" (UniqueName: \"kubernetes.io/projected/fa847739-e487-489f-8848-a97020cbc662-kube-api-access-jzw2l\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.706536 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.706482 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.709934 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.708236 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa847739-e487-489f-8848-a97020cbc662-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.710741 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.710713 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.712058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.712035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.714237 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.714211 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-ca\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.714363 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.714339 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fa847739-e487-489f-8848-a97020cbc662-hub\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.726352 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.726328 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzw2l\" (UniqueName: \"kubernetes.io/projected/fa847739-e487-489f-8848-a97020cbc662-kube-api-access-jzw2l\") pod \"cluster-proxy-proxy-agent-58f444cb84-m2fh9\" (UID: \"fa847739-e487-489f-8848-a97020cbc662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:39.765105 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:39.765078 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" Jul 20 13:02:40.666706 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:40.666629 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2116eb31_99a1_4ab5_848e_ebf5e6c4f938.slice/crio-f46185b0332fb5e5993893fa3488aa8761211e2fd86ee786c8b8f68119a309cf WatchSource:0}: Error finding container f46185b0332fb5e5993893fa3488aa8761211e2fd86ee786c8b8f68119a309cf: Status 404 returned error can't find the container with id f46185b0332fb5e5993893fa3488aa8761211e2fd86ee786c8b8f68119a309cf Jul 20 13:02:40.667928 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:40.667812 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5655955bbb-7fznj"] Jul 20 13:02:40.673862 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:40.673837 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl"] Jul 20 13:02:40.690641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:40.688279 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wgngz"] Jul 20 13:02:40.690641 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:40.689083 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f66766f_2ab9_433d_aee8_fda507d5c768.slice/crio-db480ac8080a7b7a1b106b7bfc57c4a5f15d88debeb41854c617b2e213fba6ca WatchSource:0}: Error finding container db480ac8080a7b7a1b106b7bfc57c4a5f15d88debeb41854c617b2e213fba6ca: Status 404 returned error can't find the container with id db480ac8080a7b7a1b106b7bfc57c4a5f15d88debeb41854c617b2e213fba6ca Jul 20 13:02:40.690641 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:40.690032 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d66f77fc9-zckp5"] Jul 20 13:02:40.693007 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:40.692918 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f037b6c_1ab5_453f_a697_9366f4eaebb2.slice/crio-36e35d9a02df988866ffb8265057f80fdee704d36acdfb7e6820cd1aacad6601 WatchSource:0}: Error finding container 36e35d9a02df988866ffb8265057f80fdee704d36acdfb7e6820cd1aacad6601: Status 404 returned error can't find the container with id 36e35d9a02df988866ffb8265057f80fdee704d36acdfb7e6820cd1aacad6601 Jul 20 13:02:40.693724 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:40.693701 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5107022c_b96e_4ed0_8a15_7cfb1c92ff74.slice/crio-1eb4367de74c4ab8d736f9fcb0fe5346b82a80190c8b8ab33e0db39ea3639e35 WatchSource:0}: Error finding container 1eb4367de74c4ab8d736f9fcb0fe5346b82a80190c8b8ab33e0db39ea3639e35: Status 404 returned error can't find the container with id 1eb4367de74c4ab8d736f9fcb0fe5346b82a80190c8b8ab33e0db39ea3639e35 Jul 20 13:02:40.915540 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:40.915427 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9"] Jul 20 13:02:40.918057 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:40.918025 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm"] Jul 20 13:02:40.919440 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:40.919415 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa847739_e487_489f_8848_a97020cbc662.slice/crio-05a03ea557da1add81e0ac98bbc69b97c194d8cec1f35ff8f0ae34bc22c3183c WatchSource:0}: Error finding container 05a03ea557da1add81e0ac98bbc69b97c194d8cec1f35ff8f0ae34bc22c3183c: Status 404 returned error can't find the container with id 05a03ea557da1add81e0ac98bbc69b97c194d8cec1f35ff8f0ae34bc22c3183c Jul 20 13:02:41.280740 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.280700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" event={"ID":"3b2b26a9-97cf-4c55-a817-24337c80e66b","Type":"ContainerStarted","Data":"2b630189ccc487e489b91c02934e36dde7c7b29480118505f693cd01920aab29"} Jul 20 13:02:41.282129 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.282104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xcmtb" event={"ID":"3791d792-594b-43fa-a1af-3ec0bf0aac2d","Type":"ContainerStarted","Data":"511be6f6e92b9ceaf19406504cda8ecf95d543f8f56d03acc0f82e353370fe48"} Jul 20 13:02:41.283163 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.283141 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wgngz" event={"ID":"8f037b6c-1ab5-453f-a697-9366f4eaebb2","Type":"ContainerStarted","Data":"36e35d9a02df988866ffb8265057f80fdee704d36acdfb7e6820cd1aacad6601"} Jul 20 13:02:41.284628 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.284602 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" event={"ID":"5107022c-b96e-4ed0-8a15-7cfb1c92ff74","Type":"ContainerStarted","Data":"0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8"} Jul 20 13:02:41.284723 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.284643 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" event={"ID":"5107022c-b96e-4ed0-8a15-7cfb1c92ff74","Type":"ContainerStarted","Data":"1eb4367de74c4ab8d736f9fcb0fe5346b82a80190c8b8ab33e0db39ea3639e35"} Jul 20 13:02:41.284723 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.284661 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:02:41.285771 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.285743 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" event={"ID":"fa847739-e487-489f-8848-a97020cbc662","Type":"ContainerStarted","Data":"05a03ea557da1add81e0ac98bbc69b97c194d8cec1f35ff8f0ae34bc22c3183c"} Jul 20 13:02:41.286812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.286792 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" event={"ID":"9f66766f-2ab9-433d-aee8-fda507d5c768","Type":"ContainerStarted","Data":"db480ac8080a7b7a1b106b7bfc57c4a5f15d88debeb41854c617b2e213fba6ca"} Jul 20 13:02:41.288038 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.288018 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5655955bbb-7fznj" event={"ID":"2116eb31-99a1-4ab5-848e-ebf5e6c4f938","Type":"ContainerStarted","Data":"0fd7c38fe1b0a953e305eb7b2fe15cc1662d221b2a8caea2ea3207395e0bb6a3"} Jul 20 13:02:41.288136 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.288049 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5655955bbb-7fznj" event={"ID":"2116eb31-99a1-4ab5-848e-ebf5e6c4f938","Type":"ContainerStarted","Data":"f46185b0332fb5e5993893fa3488aa8761211e2fd86ee786c8b8f68119a309cf"} Jul 20 13:02:41.317744 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.317699 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xcmtb" podStartSLOduration=65.9867381 podStartE2EDuration="1m10.317685214s" totalImagesPullingTime="4.330947114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:41.296896383 +0000 UTC m=+71.848789084" watchObservedRunningTime="2026-07-20 13:02:41.317685214 +0000 UTC m=+71.869577912" Jul 20 13:02:41.318173 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.318150 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" podStartSLOduration=70.318140617 podStartE2EDuration="1m10.318140617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:41.31718998 +0000 UTC m=+71.869082679" watchObservedRunningTime="2026-07-20 13:02:41.318140617 +0000 UTC m=+71.870033316" Jul 20 13:02:41.337086 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.337038 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5655955bbb-7fznj" podStartSLOduration=48.337023196 podStartE2EDuration="48.337023196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:41.335952665 +0000 UTC m=+71.887845379" watchObservedRunningTime="2026-07-20 13:02:41.337023196 +0000 UTC m=+71.888915895" Jul 20 13:02:41.524464 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.524427 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:41.527022 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:41.527002 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:42.294223 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:42.294142 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:42.295501 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:42.295475 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5655955bbb-7fznj" Jul 20 13:02:47.315307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:47.315205 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" event={"ID":"3b2b26a9-97cf-4c55-a817-24337c80e66b","Type":"ContainerStarted","Data":"4edaac7868fd43fc49d4bc758e41cce652b3f9e38ef0329bd55efb77da5def5a"} Jul 20 13:02:47.317294 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:47.317262 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" event={"ID":"fa847739-e487-489f-8848-a97020cbc662","Type":"ContainerStarted","Data":"ee97216d3f6cb18f9331587e4d5e21b20771bd1e2c7ad10d75b07ac37a921cfc"} Jul 20 13:02:47.317408 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:47.317313 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" event={"ID":"fa847739-e487-489f-8848-a97020cbc662","Type":"ContainerStarted","Data":"d0e69857cbcd7a12f76ad1339ec7600ebb79da0852f8960b07f9ffc364233518"} Jul 20 13:02:47.318818 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:47.318790 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" event={"ID":"9f66766f-2ab9-433d-aee8-fda507d5c768","Type":"ContainerStarted","Data":"fa4c913ecb67c7a9a922711081d190875f1065b04a8518aebc410236120dd11e"} Jul 20 13:02:48.324143 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.324104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" event={"ID":"3b2b26a9-97cf-4c55-a817-24337c80e66b","Type":"ContainerStarted","Data":"ab8dac37c8cc4b2ba1ee46624b135457a2c393255276d0ab972e63ec03588914"} Jul 20 13:02:48.325917 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.325864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wgngz" event={"ID":"8f037b6c-1ab5-453f-a697-9366f4eaebb2","Type":"ContainerStarted","Data":"ec73ac57d21688beeb53afa519777e4e08d92df12bbfac8f204b13c3e673f0b5"} Jul 20 13:02:48.326043 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.325940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wgngz" event={"ID":"8f037b6c-1ab5-453f-a697-9366f4eaebb2","Type":"ContainerStarted","Data":"c4797b449f0b9488ddb1f7259a6c4facf90a4467fff4e4e620eaf42ab74ba1fe"} Jul 20 13:02:48.326043 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.325971 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:48.327929 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.327899 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" event={"ID":"fa847739-e487-489f-8848-a97020cbc662","Type":"ContainerStarted","Data":"8a68867b4cd3794a702a81b7b533050def1d9806bc882210b4d91d6de7bdbabc"} Jul 20 13:02:48.343058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.343018 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-4v4zm" podStartSLOduration=49.307651946 podStartE2EDuration="55.343005257s" totalImagesPullingTime="6.035353311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:48.341583719 +0000 UTC m=+78.893476441" watchObservedRunningTime="2026-07-20 13:02:48.343005257 +0000 UTC m=+78.894897956" Jul 20 13:02:48.343750 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.343727 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mwnjl" podStartSLOduration=49.048132716 podStartE2EDuration="55.343717674s" totalImagesPullingTime="6.295584958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:47.339609062 +0000 UTC m=+77.891501761" watchObservedRunningTime="2026-07-20 13:02:48.343717674 +0000 UTC m=+78.895610376" Jul 20 13:02:48.361192 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.361148 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58f444cb84-m2fh9" podStartSLOduration=3.29617012 podStartE2EDuration="9.361134181s" totalImagesPullingTime="6.064964061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:48.359835668 +0000 UTC m=+78.911728369" watchObservedRunningTime="2026-07-20 13:02:48.361134181 +0000 UTC m=+78.913026903" Jul 20 13:02:48.376794 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.376751 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wgngz" podStartSLOduration=34.685602577 podStartE2EDuration="41.376738327s" totalImagesPullingTime="6.69113575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:02:48.375912681 +0000 UTC m=+78.927805402" watchObservedRunningTime="2026-07-20 13:02:48.376738327 +0000 UTC m=+78.928631025" Jul 20 13:02:48.490889 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.490837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:48.493128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.493100 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njlnn\" (UID: \"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9\") " pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:48.715046 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.715011 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rdx8c\"" Jul 20 13:02:48.723253 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.723220 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-njlnn" Jul 20 13:02:48.853516 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:48.853488 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-njlnn"] Jul 20 13:02:48.856021 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:02:48.855999 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2de0c37_7cb8_4b6c_b05b_ca279b6c6cf9.slice/crio-822ee6a7b51f7872d24745d6d20f554fe5623739668a9cd0abfd210c9fd18506 WatchSource:0}: Error finding container 822ee6a7b51f7872d24745d6d20f554fe5623739668a9cd0abfd210c9fd18506: Status 404 returned error can't find the container with id 822ee6a7b51f7872d24745d6d20f554fe5623739668a9cd0abfd210c9fd18506 Jul 20 13:02:49.332616 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:49.332580 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njlnn" event={"ID":"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9","Type":"ContainerStarted","Data":"822ee6a7b51f7872d24745d6d20f554fe5623739668a9cd0abfd210c9fd18506"} Jul 20 13:02:56.364456 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:56.364422 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njlnn" event={"ID":"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9","Type":"ContainerStarted","Data":"6f197917bcd0491c7ab60f94f2102ce28268848b4788ad2c24f0415cbb45f05f"} Jul 20 13:02:58.334725 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.334692 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wgngz" Jul 20 13:02:58.967960 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.967917 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wtlq8"] Jul 20 13:02:58.970772 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.970740 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:58.973362 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.973336 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 13:02:58.974388 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.974252 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 13:02:58.974545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.974518 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 13:02:58.974773 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.974745 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6kdv7\"" Jul 20 13:02:58.975247 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:58.975053 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 13:02:59.077594 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077481 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-sys\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077594 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-wtmp\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077594 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj9pf\" (UniqueName: \"kubernetes.io/projected/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-kube-api-access-mj9pf\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077723 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-root\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-textfile\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-metrics-client-ca\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-accelerators-collector-config\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.077869 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.077864 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178714 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178916 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178751 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-root\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178916 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-textfile\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178916 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-metrics-client-ca\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178916 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-accelerators-collector-config\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178916 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-root\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.178916 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178901 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179238 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-sys\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179238 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-wtmp\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179238 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.178995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj9pf\" (UniqueName: \"kubernetes.io/projected/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-kube-api-access-mj9pf\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179238 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:59.179123 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 13:02:59.179238 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:59.179192 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls podName:f42e17cf-729e-45d4-b9bf-63ef3c62d36e nodeName:}" failed. No retries permitted until 2026-07-20 13:02:59.679169969 +0000 UTC m=+90.231062649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls") pod "node-exporter-wtlq8" (UID: "f42e17cf-729e-45d4-b9bf-63ef3c62d36e") : secret "node-exporter-tls" not found Jul 20 13:02:59.179488 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.179259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-sys\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179488 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.179326 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-textfile\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179488 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.179396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-wtmp\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179612 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.179566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-metrics-client-ca\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.179725 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.179678 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-accelerators-collector-config\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.182081 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.182057 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.197149 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.197121 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj9pf\" (UniqueName: \"kubernetes.io/projected/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-kube-api-access-mj9pf\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.493901 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.493846 2564 patch_prober.go:28] interesting pod/image-registry-6d66f77fc9-zckp5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 13:02:59.494392 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.493937 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" podUID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 13:02:59.682712 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:02:59.682673 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:02:59.682898 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:59.682837 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 13:02:59.682970 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:02:59.682929 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls podName:f42e17cf-729e-45d4-b9bf-63ef3c62d36e nodeName:}" failed. No retries permitted until 2026-07-20 13:03:00.682912603 +0000 UTC m=+91.234805280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls") pod "node-exporter-wtlq8" (UID: "f42e17cf-729e-45d4-b9bf-63ef3c62d36e") : secret "node-exporter-tls" not found Jul 20 13:03:00.690833 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:00.690784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:03:00.693477 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:00.693448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f42e17cf-729e-45d4-b9bf-63ef3c62d36e-node-exporter-tls\") pod \"node-exporter-wtlq8\" (UID: \"f42e17cf-729e-45d4-b9bf-63ef3c62d36e\") " pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:03:00.783917 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:00.783862 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wtlq8" Jul 20 13:03:01.503496 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:01.503449 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d66f77fc9-zckp5"] Jul 20 13:03:01.508411 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:01.508383 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:03:01.723073 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:03:01.723034 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42e17cf_729e_45d4_b9bf_63ef3c62d36e.slice/crio-54664dbf0fe7c1bdc8161846af0ec8edbf0ccfbf3aef45b3d7e93c83e6666918 WatchSource:0}: Error finding container 54664dbf0fe7c1bdc8161846af0ec8edbf0ccfbf3aef45b3d7e93c83e6666918: Status 404 returned error can't find the container with id 54664dbf0fe7c1bdc8161846af0ec8edbf0ccfbf3aef45b3d7e93c83e6666918 Jul 20 13:03:02.392391 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:02.392354 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtlq8" event={"ID":"f42e17cf-729e-45d4-b9bf-63ef3c62d36e","Type":"ContainerStarted","Data":"54664dbf0fe7c1bdc8161846af0ec8edbf0ccfbf3aef45b3d7e93c83e6666918"} Jul 20 13:03:02.394490 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:02.394455 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njlnn" event={"ID":"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9","Type":"ContainerStarted","Data":"4d2419db14f2f3d0a7906a574927f94525601be85cc852a0b1fdda91d83c33fb"} Jul 20 13:03:05.410526 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:05.410490 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njlnn" event={"ID":"a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9","Type":"ContainerStarted","Data":"4add236e279504cba591b4eb29754a24e59e5308acee57780de13aacd3e39aa2"} Jul 20 13:03:05.428690 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:05.428643 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-njlnn" podStartSLOduration=33.072436017 podStartE2EDuration="49.428628364s" totalImagesPullingTime="16.356192347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:02:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:03:05.427028593 +0000 UTC m=+95.978921321" watchObservedRunningTime="2026-07-20 13:03:05.428628364 +0000 UTC m=+95.980521063" Jul 20 13:03:06.415568 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:06.415531 2564 generic.go:356] "Generic (PLEG): container finished" podID="f42e17cf-729e-45d4-b9bf-63ef3c62d36e" containerID="5645cbe1efc68ba4f79cbfbc479d9af7188e007ed1b34e98d6f9ffb514d2b1fd" exitCode=0 Jul 20 13:03:06.416056 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:06.415611 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtlq8" event={"ID":"f42e17cf-729e-45d4-b9bf-63ef3c62d36e","Type":"ContainerDied","Data":"5645cbe1efc68ba4f79cbfbc479d9af7188e007ed1b34e98d6f9ffb514d2b1fd"} Jul 20 13:03:07.267292 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:07.267258 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cwqfj" Jul 20 13:03:07.421744 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:07.421707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtlq8" event={"ID":"f42e17cf-729e-45d4-b9bf-63ef3c62d36e","Type":"ContainerStarted","Data":"06e1035480ad17d3e7a3db2d1a62ca24c0664e1aab523c15e315ed4017e8a7f0"} Jul 20 13:03:07.422138 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:07.421760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wtlq8" event={"ID":"f42e17cf-729e-45d4-b9bf-63ef3c62d36e","Type":"ContainerStarted","Data":"41142a50f3748ea1f7fac04c44fd47a27a76a59876a8a831cf3927a1417df82c"} Jul 20 13:03:07.440629 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:07.440582 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wtlq8" podStartSLOduration=5.310624833 podStartE2EDuration="9.440567496s" totalImagesPullingTime="4.063330766s" totalInitContainerRuntime="66.611897ms" isStatefulPod=true podCreationTimestamp="2026-07-20 13:02:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:03:07.438906115 +0000 UTC m=+97.990798841" watchObservedRunningTime="2026-07-20 13:03:07.440567496 +0000 UTC m=+97.992460195" Jul 20 13:03:26.525814 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.525747 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" podUID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" containerName="registry" containerID="cri-o://0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8" gracePeriod=30 Jul 20 13:03:26.825297 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.825272 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:03:26.919305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919273 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-trusted-ca\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919310 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-certificates\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919551 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919336 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-ca-trust-extracted\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919551 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919361 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919551 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919384 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-installation-pull-secrets\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919551 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919411 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-image-registry-private-configuration\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919551 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919446 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj6hl\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-kube-api-access-hj6hl\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919551 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919494 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-bound-sa-token\") pod \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\" (UID: \"5107022c-b96e-4ed0-8a15-7cfb1c92ff74\") " Jul 20 13:03:26.919847 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.919768 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:03:26.920137 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.920093 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:03:26.922044 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.922010 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-kube-api-access-hj6hl" (OuterVolumeSpecName: "kube-api-access-hj6hl") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "kube-api-access-hj6hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:03:26.922198 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.922143 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:03:26.922271 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.922230 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:03:26.922327 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.922283 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:03:26.922386 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.922353 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:03:26.927792 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:26.927769 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5107022c-b96e-4ed0-8a15-7cfb1c92ff74" (UID: "5107022c-b96e-4ed0-8a15-7cfb1c92ff74"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:03:27.021053 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021022 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-tls\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021053 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021051 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-installation-pull-secrets\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021053 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021061 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-image-registry-private-configuration\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021266 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021071 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj6hl\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-kube-api-access-hj6hl\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021266 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021079 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-bound-sa-token\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021266 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021087 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-trusted-ca\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021266 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021095 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-registry-certificates\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.021266 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.021103 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5107022c-b96e-4ed0-8a15-7cfb1c92ff74-ca-trust-extracted\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:03:27.494824 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.494786 2564 generic.go:356] "Generic (PLEG): container finished" podID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" containerID="0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8" exitCode=0 Jul 20 13:03:27.495038 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.494845 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" Jul 20 13:03:27.495038 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.494864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" event={"ID":"5107022c-b96e-4ed0-8a15-7cfb1c92ff74","Type":"ContainerDied","Data":"0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8"} Jul 20 13:03:27.495038 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.494927 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d66f77fc9-zckp5" event={"ID":"5107022c-b96e-4ed0-8a15-7cfb1c92ff74","Type":"ContainerDied","Data":"1eb4367de74c4ab8d736f9fcb0fe5346b82a80190c8b8ab33e0db39ea3639e35"} Jul 20 13:03:27.495038 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.494946 2564 scope.go:117] "RemoveContainer" containerID="0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8" Jul 20 13:03:27.503751 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.503729 2564 scope.go:117] "RemoveContainer" containerID="0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8" Jul 20 13:03:27.504019 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:03:27.503999 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8\": container with ID starting with 0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8 not found: ID does not exist" containerID="0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8" Jul 20 13:03:27.504100 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.504025 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8"} err="failed to get container status \"0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8\": rpc error: code = NotFound desc = could not find container \"0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8\": container with ID starting with 0c2c63bcbaf13a7cc562b00c8efaf312a35c68c36b7db468d11df11d148f38b8 not found: ID does not exist" Jul 20 13:03:27.520523 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.517543 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d66f77fc9-zckp5"] Jul 20 13:03:27.524859 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.524827 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d66f77fc9-zckp5"] Jul 20 13:03:27.934894 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:27.934799 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" path="/var/lib/kubelet/pods/5107022c-b96e-4ed0-8a15-7cfb1c92ff74/volumes" Jul 20 13:03:31.510680 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:31.510644 2564 generic.go:356] "Generic (PLEG): container finished" podID="50bbd98b-5610-4ad4-a413-3e4eec07581e" containerID="eb13549444b17136f7df7887922194162932dcf56715d251ad03ae30eab8d6b7" exitCode=0 Jul 20 13:03:31.511180 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:31.510698 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" event={"ID":"50bbd98b-5610-4ad4-a413-3e4eec07581e","Type":"ContainerDied","Data":"eb13549444b17136f7df7887922194162932dcf56715d251ad03ae30eab8d6b7"} Jul 20 13:03:31.511272 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:31.511199 2564 scope.go:117] "RemoveContainer" containerID="eb13549444b17136f7df7887922194162932dcf56715d251ad03ae30eab8d6b7" Jul 20 13:03:32.516604 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:32.516571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jnvhs" event={"ID":"50bbd98b-5610-4ad4-a413-3e4eec07581e","Type":"ContainerStarted","Data":"a4f902a4222bde237b1e522d2a6ea8e0ab19bd0be23e57ba749808f2afebbe40"} Jul 20 13:03:37.535850 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:37.535817 2564 generic.go:356] "Generic (PLEG): container finished" podID="85eff64d-ea97-41ab-a625-39ae59200b81" containerID="8dc880f51fa611a37f37a0070bc4b3e5da62b4a2a3ab825767ea2858dd7941bc" exitCode=0 Jul 20 13:03:37.536244 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:37.535911 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" event={"ID":"85eff64d-ea97-41ab-a625-39ae59200b81","Type":"ContainerDied","Data":"8dc880f51fa611a37f37a0070bc4b3e5da62b4a2a3ab825767ea2858dd7941bc"} Jul 20 13:03:37.536285 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:37.536263 2564 scope.go:117] "RemoveContainer" containerID="8dc880f51fa611a37f37a0070bc4b3e5da62b4a2a3ab825767ea2858dd7941bc" Jul 20 13:03:38.540892 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:38.540847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-727ql" event={"ID":"85eff64d-ea97-41ab-a625-39ae59200b81","Type":"ContainerStarted","Data":"6b27ee85fb3149dd9ad265cd2a1fa047e658b593eae45fb2de85466cad95c98e"} Jul 20 13:03:47.581480 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:47.581447 2564 generic.go:356] "Generic (PLEG): container finished" podID="031c9773-aee4-42f0-ab1e-68925940613b" containerID="ec6f0a4495dee0b3d6a07deb1c3fbcaae8659fd34b87ea245ddfd2c727188a97" exitCode=0 Jul 20 13:03:47.581935 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:47.581519 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" event={"ID":"031c9773-aee4-42f0-ab1e-68925940613b","Type":"ContainerDied","Data":"ec6f0a4495dee0b3d6a07deb1c3fbcaae8659fd34b87ea245ddfd2c727188a97"} Jul 20 13:03:47.581935 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:47.581812 2564 scope.go:117] "RemoveContainer" containerID="ec6f0a4495dee0b3d6a07deb1c3fbcaae8659fd34b87ea245ddfd2c727188a97" Jul 20 13:03:48.586528 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:03:48.586491 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cfqrm" event={"ID":"031c9773-aee4-42f0-ab1e-68925940613b","Type":"ContainerStarted","Data":"abd979e3d2d0b813356637487a370b75e265cbc54f2ee0344d1973b19720e8a6"} Jul 20 13:06:29.806963 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:29.806935 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:06:29.811007 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:29.810983 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:06:29.811365 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:29.811350 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:06:29.815594 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:29.815570 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:06:29.818222 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:29.818191 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 13:06:34.414090 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.414059 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w"] Jul 20 13:06:34.416414 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.414312 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" containerName="registry" Jul 20 13:06:34.416414 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.414322 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" containerName="registry" Jul 20 13:06:34.416414 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.414375 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5107022c-b96e-4ed0-8a15-7cfb1c92ff74" containerName="registry" Jul 20 13:06:34.417313 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.417298 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.421101 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.421077 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:06:34.421222 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.421075 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 20 13:06:34.421222 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.421119 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-j2knl\"" Jul 20 13:06:34.449646 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.449613 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w"] Jul 20 13:06:34.517891 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.517838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzbf6\" (UniqueName: \"kubernetes.io/projected/168f3264-f619-4252-9666-536eeff463c3-kube-api-access-lzbf6\") pod \"cert-manager-operator-controller-manager-55c444bf86-5gx4w\" (UID: \"168f3264-f619-4252-9666-536eeff463c3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.518038 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.517903 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168f3264-f619-4252-9666-536eeff463c3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5gx4w\" (UID: \"168f3264-f619-4252-9666-536eeff463c3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.618837 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.618791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzbf6\" (UniqueName: \"kubernetes.io/projected/168f3264-f619-4252-9666-536eeff463c3-kube-api-access-lzbf6\") pod \"cert-manager-operator-controller-manager-55c444bf86-5gx4w\" (UID: \"168f3264-f619-4252-9666-536eeff463c3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.619032 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.618851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168f3264-f619-4252-9666-536eeff463c3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5gx4w\" (UID: \"168f3264-f619-4252-9666-536eeff463c3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.619197 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.619179 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168f3264-f619-4252-9666-536eeff463c3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5gx4w\" (UID: \"168f3264-f619-4252-9666-536eeff463c3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.628693 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.628671 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzbf6\" (UniqueName: \"kubernetes.io/projected/168f3264-f619-4252-9666-536eeff463c3-kube-api-access-lzbf6\") pod \"cert-manager-operator-controller-manager-55c444bf86-5gx4w\" (UID: \"168f3264-f619-4252-9666-536eeff463c3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.726799 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.726763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" Jul 20 13:06:34.858743 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.858715 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w"] Jul 20 13:06:34.863973 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:06:34.863943 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168f3264_f619_4252_9666_536eeff463c3.slice/crio-a34dc036d483417adc2107940c1636fc6f4611a06b08d9f7d90531959b43a8c5 WatchSource:0}: Error finding container a34dc036d483417adc2107940c1636fc6f4611a06b08d9f7d90531959b43a8c5: Status 404 returned error can't find the container with id a34dc036d483417adc2107940c1636fc6f4611a06b08d9f7d90531959b43a8c5 Jul 20 13:06:34.871514 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:34.871486 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 13:06:35.183376 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:35.183288 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" event={"ID":"168f3264-f619-4252-9666-536eeff463c3","Type":"ContainerStarted","Data":"a34dc036d483417adc2107940c1636fc6f4611a06b08d9f7d90531959b43a8c5"} Jul 20 13:06:38.199281 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:38.199243 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" event={"ID":"168f3264-f619-4252-9666-536eeff463c3","Type":"ContainerStarted","Data":"1a0f4ebc0c1e382e5a91c66e114bbc315ec42ea734e65af58aa1e94909918db6"} Jul 20 13:06:38.244928 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:38.244856 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5gx4w" podStartSLOduration=1.56080328 podStartE2EDuration="4.244835748s" totalImagesPullingTime="2.684032468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:06:38.238388572 +0000 UTC m=+308.790281270" watchObservedRunningTime="2026-07-20 13:06:38.244835748 +0000 UTC m=+308.796728449" Jul 20 13:06:39.085578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.085547 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq"] Jul 20 13:06:39.089057 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.089040 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.093778 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.093752 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 13:06:39.093778 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.093765 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 13:06:39.093971 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.093826 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d2nvl\"" Jul 20 13:06:39.109476 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.109457 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq"] Jul 20 13:06:39.154987 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.154962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.155130 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.154998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdf9k\" (UniqueName: \"kubernetes.io/projected/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-kube-api-access-qdf9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.155130 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.155048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.256160 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.256129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdf9k\" (UniqueName: \"kubernetes.io/projected/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-kube-api-access-qdf9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.256530 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.256185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.256530 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.256225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.256639 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.256617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.256679 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.256649 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.265265 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.265240 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdf9k\" (UniqueName: \"kubernetes.io/projected/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-kube-api-access-qdf9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.398324 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.398248 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:39.533091 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:39.533059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq"] Jul 20 13:06:39.535895 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:06:39.535857 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03dfb93f_b21a_4e2d_9816_bf5ed8090c98.slice/crio-f6080321031f7306b12cca65e42347a8e3da99919ff0241c0df7b40cee4c8371 WatchSource:0}: Error finding container f6080321031f7306b12cca65e42347a8e3da99919ff0241c0df7b40cee4c8371: Status 404 returned error can't find the container with id f6080321031f7306b12cca65e42347a8e3da99919ff0241c0df7b40cee4c8371 Jul 20 13:06:40.208689 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:40.208652 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" event={"ID":"03dfb93f-b21a-4e2d-9816-bf5ed8090c98","Type":"ContainerStarted","Data":"f6080321031f7306b12cca65e42347a8e3da99919ff0241c0df7b40cee4c8371"} Jul 20 13:06:47.242228 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:47.242195 2564 generic.go:356] "Generic (PLEG): container finished" podID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerID="ed0c8b9720e9715924e76bdbf280ce474f2c59dd94f3d82522ef12b67c85ef5e" exitCode=0 Jul 20 13:06:47.242631 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:47.242279 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" event={"ID":"03dfb93f-b21a-4e2d-9816-bf5ed8090c98","Type":"ContainerDied","Data":"ed0c8b9720e9715924e76bdbf280ce474f2c59dd94f3d82522ef12b67c85ef5e"} Jul 20 13:06:50.258548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:50.258514 2564 generic.go:356] "Generic (PLEG): container finished" podID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerID="b18dc0fb634ca8bd20dfc9c831eea8a612cb34f6f8dc8b2e4b329e892bf09fff" exitCode=0 Jul 20 13:06:50.258958 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:50.258599 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" event={"ID":"03dfb93f-b21a-4e2d-9816-bf5ed8090c98","Type":"ContainerDied","Data":"b18dc0fb634ca8bd20dfc9c831eea8a612cb34f6f8dc8b2e4b329e892bf09fff"} Jul 20 13:06:58.298162 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:58.298122 2564 generic.go:356] "Generic (PLEG): container finished" podID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerID="c67d940b464311352ca1a532262f07c62beab6be5e14b7d784163688c526360c" exitCode=0 Jul 20 13:06:58.298545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:58.298203 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" event={"ID":"03dfb93f-b21a-4e2d-9816-bf5ed8090c98","Type":"ContainerDied","Data":"c67d940b464311352ca1a532262f07c62beab6be5e14b7d784163688c526360c"} Jul 20 13:06:59.424273 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.424249 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:06:59.531964 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.531934 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdf9k\" (UniqueName: \"kubernetes.io/projected/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-kube-api-access-qdf9k\") pod \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " Jul 20 13:06:59.532112 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.531992 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-util\") pod \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " Jul 20 13:06:59.532112 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.532020 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-bundle\") pod \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\" (UID: \"03dfb93f-b21a-4e2d-9816-bf5ed8090c98\") " Jul 20 13:06:59.532445 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.532416 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-bundle" (OuterVolumeSpecName: "bundle") pod "03dfb93f-b21a-4e2d-9816-bf5ed8090c98" (UID: "03dfb93f-b21a-4e2d-9816-bf5ed8090c98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:06:59.534200 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.534172 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-kube-api-access-qdf9k" (OuterVolumeSpecName: "kube-api-access-qdf9k") pod "03dfb93f-b21a-4e2d-9816-bf5ed8090c98" (UID: "03dfb93f-b21a-4e2d-9816-bf5ed8090c98"). InnerVolumeSpecName "kube-api-access-qdf9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:06:59.536085 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.536064 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-util" (OuterVolumeSpecName: "util") pod "03dfb93f-b21a-4e2d-9816-bf5ed8090c98" (UID: "03dfb93f-b21a-4e2d-9816-bf5ed8090c98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:06:59.632571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.632507 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdf9k\" (UniqueName: \"kubernetes.io/projected/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-kube-api-access-qdf9k\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:06:59.632571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.632530 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-util\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:06:59.632571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:06:59.632540 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03dfb93f-b21a-4e2d-9816-bf5ed8090c98-bundle\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:00.308946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:00.308837 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" event={"ID":"03dfb93f-b21a-4e2d-9816-bf5ed8090c98","Type":"ContainerDied","Data":"f6080321031f7306b12cca65e42347a8e3da99919ff0241c0df7b40cee4c8371"} Jul 20 13:07:00.308946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:00.308855 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4m9lq" Jul 20 13:07:00.308946 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:00.308897 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6080321031f7306b12cca65e42347a8e3da99919ff0241c0df7b40cee4c8371" Jul 20 13:07:08.878664 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878625 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg"] Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878892 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="pull" Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878902 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="pull" Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878913 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="extract" Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878919 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="extract" Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878926 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="util" Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878932 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="util" Jul 20 13:07:08.879128 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.878990 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="03dfb93f-b21a-4e2d-9816-bf5ed8090c98" containerName="extract" Jul 20 13:07:08.883919 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.883898 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:08.886750 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.886726 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 13:07:08.887185 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.887163 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 13:07:08.888024 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.888008 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d2nvl\"" Jul 20 13:07:08.891736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:08.891714 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg"] Jul 20 13:07:09.008016 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.007983 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.008182 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.008032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl4vw\" (UniqueName: \"kubernetes.io/projected/3355c800-020b-4172-80fa-c8dd26a2fbfd-kube-api-access-gl4vw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.008182 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.008059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.108556 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.108519 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.108694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.108565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl4vw\" (UniqueName: \"kubernetes.io/projected/3355c800-020b-4172-80fa-c8dd26a2fbfd-kube-api-access-gl4vw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.108694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.108623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.109024 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.109002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.109068 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.109010 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.122211 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.122182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl4vw\" (UniqueName: \"kubernetes.io/projected/3355c800-020b-4172-80fa-c8dd26a2fbfd-kube-api-access-gl4vw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.193786 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.193761 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:09.325056 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:09.324999 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3355c800_020b_4172_80fa_c8dd26a2fbfd.slice/crio-f97b97a22be9a8cdc89c0123719beba510937b0072a4642dd9ee6759efaf6897 WatchSource:0}: Error finding container f97b97a22be9a8cdc89c0123719beba510937b0072a4642dd9ee6759efaf6897: Status 404 returned error can't find the container with id f97b97a22be9a8cdc89c0123719beba510937b0072a4642dd9ee6759efaf6897 Jul 20 13:07:09.327042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.327016 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg"] Jul 20 13:07:09.345770 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:09.345745 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" event={"ID":"3355c800-020b-4172-80fa-c8dd26a2fbfd","Type":"ContainerStarted","Data":"f97b97a22be9a8cdc89c0123719beba510937b0072a4642dd9ee6759efaf6897"} Jul 20 13:07:10.355993 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:10.355952 2564 generic.go:356] "Generic (PLEG): container finished" podID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerID="e2bf60cc696b7724e15effa75371efb77da36686813277a11c4d0bef617b43c8" exitCode=0 Jul 20 13:07:10.356357 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:10.356008 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" event={"ID":"3355c800-020b-4172-80fa-c8dd26a2fbfd","Type":"ContainerDied","Data":"e2bf60cc696b7724e15effa75371efb77da36686813277a11c4d0bef617b43c8"} Jul 20 13:07:11.361532 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:11.361449 2564 generic.go:356] "Generic (PLEG): container finished" podID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerID="2233726a82f052cd965a9b0ee1880f3dff53cc839f3b1ab6e62f4fb409947899" exitCode=0 Jul 20 13:07:11.361532 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:11.361515 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" event={"ID":"3355c800-020b-4172-80fa-c8dd26a2fbfd","Type":"ContainerDied","Data":"2233726a82f052cd965a9b0ee1880f3dff53cc839f3b1ab6e62f4fb409947899"} Jul 20 13:07:12.367625 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:12.367590 2564 generic.go:356] "Generic (PLEG): container finished" podID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerID="8623d2be57aeb75453e3de3b1ac24d3274c47e99006eb9df13b4f9788a452426" exitCode=0 Jul 20 13:07:12.368042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:12.367689 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" event={"ID":"3355c800-020b-4172-80fa-c8dd26a2fbfd","Type":"ContainerDied","Data":"8623d2be57aeb75453e3de3b1ac24d3274c47e99006eb9df13b4f9788a452426"} Jul 20 13:07:13.495250 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.495224 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:13.647207 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.647115 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-bundle\") pod \"3355c800-020b-4172-80fa-c8dd26a2fbfd\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " Jul 20 13:07:13.647207 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.647204 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl4vw\" (UniqueName: \"kubernetes.io/projected/3355c800-020b-4172-80fa-c8dd26a2fbfd-kube-api-access-gl4vw\") pod \"3355c800-020b-4172-80fa-c8dd26a2fbfd\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " Jul 20 13:07:13.647403 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.647251 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-util\") pod \"3355c800-020b-4172-80fa-c8dd26a2fbfd\" (UID: \"3355c800-020b-4172-80fa-c8dd26a2fbfd\") " Jul 20 13:07:13.647863 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.647826 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-bundle" (OuterVolumeSpecName: "bundle") pod "3355c800-020b-4172-80fa-c8dd26a2fbfd" (UID: "3355c800-020b-4172-80fa-c8dd26a2fbfd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:13.649431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.649403 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3355c800-020b-4172-80fa-c8dd26a2fbfd-kube-api-access-gl4vw" (OuterVolumeSpecName: "kube-api-access-gl4vw") pod "3355c800-020b-4172-80fa-c8dd26a2fbfd" (UID: "3355c800-020b-4172-80fa-c8dd26a2fbfd"). InnerVolumeSpecName "kube-api-access-gl4vw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:07:13.652469 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.652444 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-util" (OuterVolumeSpecName: "util") pod "3355c800-020b-4172-80fa-c8dd26a2fbfd" (UID: "3355c800-020b-4172-80fa-c8dd26a2fbfd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:13.748036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.747997 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl4vw\" (UniqueName: \"kubernetes.io/projected/3355c800-020b-4172-80fa-c8dd26a2fbfd-kube-api-access-gl4vw\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.748036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.748031 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-util\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:13.748036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:13.748041 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3355c800-020b-4172-80fa-c8dd26a2fbfd-bundle\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:14.377794 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:14.377710 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" event={"ID":"3355c800-020b-4172-80fa-c8dd26a2fbfd","Type":"ContainerDied","Data":"f97b97a22be9a8cdc89c0123719beba510937b0072a4642dd9ee6759efaf6897"} Jul 20 13:07:14.377794 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:14.377739 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbwvlg" Jul 20 13:07:14.377794 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:14.377759 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f97b97a22be9a8cdc89c0123719beba510937b0072a4642dd9ee6759efaf6897" Jul 20 13:07:22.434622 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434542 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn"] Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434807 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="pull" Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434817 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="pull" Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434828 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="util" Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434833 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="util" Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434845 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="extract" Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434850 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="extract" Jul 20 13:07:22.435017 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.434916 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3355c800-020b-4172-80fa-c8dd26a2fbfd" containerName="extract" Jul 20 13:07:22.436987 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.436971 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.439702 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.439681 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 13:07:22.439851 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.439784 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 13:07:22.439851 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.439785 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 13:07:22.440066 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.440049 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s8pp8\"" Jul 20 13:07:22.440134 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.440084 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 13:07:22.454035 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.454014 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn"] Jul 20 13:07:22.615993 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.615961 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.615993 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.616000 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-manifests\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.616211 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.616027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp4gc\" (UniqueName: \"kubernetes.io/projected/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-kube-api-access-qp4gc\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.616211 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.616095 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.616211 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.616131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-tmp\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.717399 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.717365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.717399 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.717398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-manifests\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.717639 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.717416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp4gc\" (UniqueName: \"kubernetes.io/projected/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-kube-api-access-qp4gc\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.717639 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.717450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.717639 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.717478 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-tmp\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.717857 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.717745 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-manifests\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.719728 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.719701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-tmp\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.719986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.719967 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.720154 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.720135 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.726238 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.726212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp4gc\" (UniqueName: \"kubernetes.io/projected/61f1ea27-3445-4f8f-9f59-bfa7978b15c9-kube-api-access-qp4gc\") pod \"opendatahub-operator-controller-manager-85766d874c-dbdzn\" (UID: \"61f1ea27-3445-4f8f-9f59-bfa7978b15c9\") " pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.746548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.746530 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:22.897389 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:22.897357 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn"] Jul 20 13:07:22.898776 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:22.898752 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f1ea27_3445_4f8f_9f59_bfa7978b15c9.slice/crio-0a3362f8b9000bc098d382743e11f7a98048dbe8be534173d70b825cfef3a489 WatchSource:0}: Error finding container 0a3362f8b9000bc098d382743e11f7a98048dbe8be534173d70b825cfef3a489: Status 404 returned error can't find the container with id 0a3362f8b9000bc098d382743e11f7a98048dbe8be534173d70b825cfef3a489 Jul 20 13:07:23.422579 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:23.422543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" event={"ID":"61f1ea27-3445-4f8f-9f59-bfa7978b15c9","Type":"ContainerStarted","Data":"0a3362f8b9000bc098d382743e11f7a98048dbe8be534173d70b825cfef3a489"} Jul 20 13:07:26.440362 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:26.440325 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" event={"ID":"61f1ea27-3445-4f8f-9f59-bfa7978b15c9","Type":"ContainerStarted","Data":"7482fa76a8ccf88fd91a30e8bd16c9a1f443725c3ba45b232e31965581a0865b"} Jul 20 13:07:27.445541 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:27.445508 2564 generic.go:356] "Generic (PLEG): container finished" podID="61f1ea27-3445-4f8f-9f59-bfa7978b15c9" containerID="7482fa76a8ccf88fd91a30e8bd16c9a1f443725c3ba45b232e31965581a0865b" exitCode=0 Jul 20 13:07:27.445898 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:27.445547 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" event={"ID":"61f1ea27-3445-4f8f-9f59-bfa7978b15c9","Type":"ContainerDied","Data":"7482fa76a8ccf88fd91a30e8bd16c9a1f443725c3ba45b232e31965581a0865b"} Jul 20 13:07:29.455339 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:29.455305 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" event={"ID":"61f1ea27-3445-4f8f-9f59-bfa7978b15c9","Type":"ContainerStarted","Data":"9a06ff0bbe64fe7c76906b37276cdc2cfa31307c0d4f9dd5e54ebdfc33723142"} Jul 20 13:07:29.455799 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:29.455416 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:29.479665 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:29.479569 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" podStartSLOduration=1.649620898 podStartE2EDuration="7.479555435s" totalImagesPullingTime="4.740813954s" totalInitContainerRuntime="1.089120583s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:07:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:07:29.477162745 +0000 UTC m=+360.029055444" watchObservedRunningTime="2026-07-20 13:07:29.479555435 +0000 UTC m=+360.031448133" Jul 20 13:07:40.461886 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:40.461841 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85766d874c-dbdzn" Jul 20 13:07:42.782512 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.782477 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg"] Jul 20 13:07:42.790333 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.790301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.794628 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.794597 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 13:07:42.794628 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.794619 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 13:07:42.794853 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.794604 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-69lqz\"" Jul 20 13:07:42.796021 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.795993 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg"] Jul 20 13:07:42.860769 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.860737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-tmp\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.860983 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.860779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brvz2\" (UniqueName: \"kubernetes.io/projected/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-kube-api-access-brvz2\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.860983 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.860810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-tls-certs\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.900383 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.900354 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2xtgl"] Jul 20 13:07:42.902561 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.902543 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:42.905018 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.904995 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4lhx2\"" Jul 20 13:07:42.905126 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.905028 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 20 13:07:42.914434 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.914414 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2xtgl"] Jul 20 13:07:42.961794 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.961766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-tmp\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.961953 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.961802 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0590f04d-997c-474a-91ad-8b13d74f296f-cert\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:42.961953 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.961828 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brvz2\" (UniqueName: \"kubernetes.io/projected/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-kube-api-access-brvz2\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.961953 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.961856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2mp7\" (UniqueName: \"kubernetes.io/projected/0590f04d-997c-474a-91ad-8b13d74f296f-kube-api-access-m2mp7\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:42.962089 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.962068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-tls-certs\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.964036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.964014 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-tmp\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.964164 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.964147 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-tls-certs\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:42.969974 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:42.969951 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brvz2\" (UniqueName: \"kubernetes.io/projected/f4b86a98-206e-4b56-8bbc-c427af1bb7e1-kube-api-access-brvz2\") pod \"kube-auth-proxy-7bb65f5cb7-z9wmg\" (UID: \"f4b86a98-206e-4b56-8bbc-c427af1bb7e1\") " pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:43.063233 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.063150 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2mp7\" (UniqueName: \"kubernetes.io/projected/0590f04d-997c-474a-91ad-8b13d74f296f-kube-api-access-m2mp7\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:43.063233 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.063210 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0590f04d-997c-474a-91ad-8b13d74f296f-cert\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:43.063445 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:07:43.063319 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 20 13:07:43.063445 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:07:43.063392 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590f04d-997c-474a-91ad-8b13d74f296f-cert podName:0590f04d-997c-474a-91ad-8b13d74f296f nodeName:}" failed. No retries permitted until 2026-07-20 13:07:43.563373093 +0000 UTC m=+374.115265783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0590f04d-997c-474a-91ad-8b13d74f296f-cert") pod "odh-model-controller-7998c67dc4-2xtgl" (UID: "0590f04d-997c-474a-91ad-8b13d74f296f") : secret "odh-model-controller-webhook-cert" not found Jul 20 13:07:43.071834 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.071814 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2mp7\" (UniqueName: \"kubernetes.io/projected/0590f04d-997c-474a-91ad-8b13d74f296f-kube-api-access-m2mp7\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:43.100714 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.100691 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" Jul 20 13:07:43.225156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.225134 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg"] Jul 20 13:07:43.227455 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:43.227427 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b86a98_206e_4b56_8bbc_c427af1bb7e1.slice/crio-1fa134268487e88d730c43217e5f22706c7fc23bc92f3f5d86eef6b8968d6bf3 WatchSource:0}: Error finding container 1fa134268487e88d730c43217e5f22706c7fc23bc92f3f5d86eef6b8968d6bf3: Status 404 returned error can't find the container with id 1fa134268487e88d730c43217e5f22706c7fc23bc92f3f5d86eef6b8968d6bf3 Jul 20 13:07:43.440505 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.440470 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jq7bz"] Jul 20 13:07:43.443165 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.443149 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:43.446133 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.446108 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 20 13:07:43.462120 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.462096 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jq7bz"] Jul 20 13:07:43.466736 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.466710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcqll\" (UniqueName: \"kubernetes.io/projected/eeb1feda-4d8e-4787-916f-df9659c3ca6d-kube-api-access-zcqll\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:43.466835 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.466777 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb1feda-4d8e-4787-916f-df9659c3ca6d-tls-certs\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:43.515158 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.515125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" event={"ID":"f4b86a98-206e-4b56-8bbc-c427af1bb7e1","Type":"ContainerStarted","Data":"1fa134268487e88d730c43217e5f22706c7fc23bc92f3f5d86eef6b8968d6bf3"} Jul 20 13:07:43.567669 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.567638 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb1feda-4d8e-4787-916f-df9659c3ca6d-tls-certs\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:43.567790 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.567687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0590f04d-997c-474a-91ad-8b13d74f296f-cert\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:43.567790 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.567741 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcqll\" (UniqueName: \"kubernetes.io/projected/eeb1feda-4d8e-4787-916f-df9659c3ca6d-kube-api-access-zcqll\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:43.567860 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:07:43.567791 2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 20 13:07:43.567860 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:07:43.567841 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeb1feda-4d8e-4787-916f-df9659c3ca6d-tls-certs podName:eeb1feda-4d8e-4787-916f-df9659c3ca6d nodeName:}" failed. No retries permitted until 2026-07-20 13:07:44.067826147 +0000 UTC m=+374.619718827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/eeb1feda-4d8e-4787-916f-df9659c3ca6d-tls-certs") pod "model-serving-api-54d89bd5fc-jq7bz" (UID: "eeb1feda-4d8e-4787-916f-df9659c3ca6d") : secret "model-serving-api-tls" not found Jul 20 13:07:43.570141 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.570123 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0590f04d-997c-474a-91ad-8b13d74f296f-cert\") pod \"odh-model-controller-7998c67dc4-2xtgl\" (UID: \"0590f04d-997c-474a-91ad-8b13d74f296f\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:43.579771 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.579748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcqll\" (UniqueName: \"kubernetes.io/projected/eeb1feda-4d8e-4787-916f-df9659c3ca6d-kube-api-access-zcqll\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:43.813004 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.812916 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:43.979650 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:43.979602 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2xtgl"] Jul 20 13:07:43.981941 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:43.981915 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0590f04d_997c_474a_91ad_8b13d74f296f.slice/crio-e8a77ded85e6bb68d2a479fe650299067d3c02433199c54ed7b3a9bcd79681d7 WatchSource:0}: Error finding container e8a77ded85e6bb68d2a479fe650299067d3c02433199c54ed7b3a9bcd79681d7: Status 404 returned error can't find the container with id e8a77ded85e6bb68d2a479fe650299067d3c02433199c54ed7b3a9bcd79681d7 Jul 20 13:07:44.049902 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.049841 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb"] Jul 20 13:07:44.052390 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.052367 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.056127 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.056105 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 20 13:07:44.056237 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.056144 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 20 13:07:44.056237 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.056192 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 20 13:07:44.056509 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.056493 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7ctwb\"" Jul 20 13:07:44.056579 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.056519 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 13:07:44.066959 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.066907 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb"] Jul 20 13:07:44.075449 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075426 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fab8b8-969d-4594-a8c9-5571a74558e4-local-certs\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-token\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-cacerts\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075648 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg2hp\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-kube-api-access-lg2hp\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075648 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075729 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075674 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075766 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.075808 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.075780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb1feda-4d8e-4787-916f-df9659c3ca6d-tls-certs\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:44.078459 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.078436 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb1feda-4d8e-4787-916f-df9659c3ca6d-tls-certs\") pod \"model-serving-api-54d89bd5fc-jq7bz\" (UID: \"eeb1feda-4d8e-4787-916f-df9659c3ca6d\") " pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:44.176390 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176357 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fab8b8-969d-4594-a8c9-5571a74558e4-local-certs\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.176390 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-token\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.176633 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-cacerts\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.176633 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg2hp\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-kube-api-access-lg2hp\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.176739 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.176739 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176697 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.176965 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.176864 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.177471 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.177435 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.179120 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.179089 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-cacerts\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.179213 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.179162 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fab8b8-969d-4594-a8c9-5571a74558e4-local-certs\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.179323 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.179299 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.179590 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.179574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.185590 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.185565 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-token\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.185990 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.185964 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg2hp\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-kube-api-access-lg2hp\") pod \"istiod-openshift-gateway-7574459656-hvmvb\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.353763 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.353686 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:44.361975 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.361955 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:44.506724 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.506680 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jq7bz"] Jul 20 13:07:44.509011 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:44.508983 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb1feda_4d8e_4787_916f_df9659c3ca6d.slice/crio-050764cba8e98127bcd191a7becdd616b8bfc098a85ba9b70b982226c1ce0c3f WatchSource:0}: Error finding container 050764cba8e98127bcd191a7becdd616b8bfc098a85ba9b70b982226c1ce0c3f: Status 404 returned error can't find the container with id 050764cba8e98127bcd191a7becdd616b8bfc098a85ba9b70b982226c1ce0c3f Jul 20 13:07:44.523840 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.523800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" event={"ID":"0590f04d-997c-474a-91ad-8b13d74f296f","Type":"ContainerStarted","Data":"e8a77ded85e6bb68d2a479fe650299067d3c02433199c54ed7b3a9bcd79681d7"} Jul 20 13:07:44.525036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.525010 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" event={"ID":"eeb1feda-4d8e-4787-916f-df9659c3ca6d","Type":"ContainerStarted","Data":"050764cba8e98127bcd191a7becdd616b8bfc098a85ba9b70b982226c1ce0c3f"} Jul 20 13:07:44.535592 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:44.535511 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb"] Jul 20 13:07:44.538053 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:44.538028 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fab8b8_969d_4594_a8c9_5571a74558e4.slice/crio-ff0e9b5a816c737bc973cf0f41f5b874cb2557386a8b57a07fa709268ac9503c WatchSource:0}: Error finding container ff0e9b5a816c737bc973cf0f41f5b874cb2557386a8b57a07fa709268ac9503c: Status 404 returned error can't find the container with id ff0e9b5a816c737bc973cf0f41f5b874cb2557386a8b57a07fa709268ac9503c Jul 20 13:07:45.534266 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:45.534213 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" event={"ID":"a4fab8b8-969d-4594-a8c9-5571a74558e4","Type":"ContainerStarted","Data":"ff0e9b5a816c737bc973cf0f41f5b874cb2557386a8b57a07fa709268ac9503c"} Jul 20 13:07:49.082149 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.082118 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 13:07:49.082491 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.082181 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 13:07:49.557239 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.557188 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" event={"ID":"a4fab8b8-969d-4594-a8c9-5571a74558e4","Type":"ContainerStarted","Data":"0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e"} Jul 20 13:07:49.557428 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.557399 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:49.558858 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.558836 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" event={"ID":"0590f04d-997c-474a-91ad-8b13d74f296f","Type":"ContainerStarted","Data":"61909c59d16e44504a30e28510a46f24c1cece9c2a7c7b53502b14f7fe31833f"} Jul 20 13:07:49.558997 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.558925 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:07:49.560332 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.560313 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" event={"ID":"f4b86a98-206e-4b56-8bbc-c427af1bb7e1","Type":"ContainerStarted","Data":"ae0c34f142a1a6b73b0b59cf049b8655daf8c9891d5454deb976f7cf97c23e7a"} Jul 20 13:07:49.561838 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.561817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" event={"ID":"eeb1feda-4d8e-4787-916f-df9659c3ca6d","Type":"ContainerStarted","Data":"ce44232810a43da707ce3f74f2b3b541a952b719d1e6baeb762668c4264aedf4"} Jul 20 13:07:49.561945 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.561910 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:07:49.601052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.601009 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" podStartSLOduration=1.058975228 podStartE2EDuration="5.600997015s" totalImagesPullingTime="4.542021787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:07:49.598335097 +0000 UTC m=+380.150227795" watchObservedRunningTime="2026-07-20 13:07:49.600997015 +0000 UTC m=+380.152889691" Jul 20 13:07:49.627746 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.627675 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bb65f5cb7-z9wmg" podStartSLOduration=1.781217028 podStartE2EDuration="7.627651518s" totalImagesPullingTime="5.84643449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:07:49.626165759 +0000 UTC m=+380.178058452" watchObservedRunningTime="2026-07-20 13:07:49.627651518 +0000 UTC m=+380.179544265" Jul 20 13:07:49.652603 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.652554 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" podStartSLOduration=2.503491947 podStartE2EDuration="7.652537047s" totalImagesPullingTime="5.1490451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:07:49.649802396 +0000 UTC m=+380.201695096" watchObservedRunningTime="2026-07-20 13:07:49.652537047 +0000 UTC m=+380.204429745" Jul 20 13:07:49.670683 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:49.670633 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" podStartSLOduration=2.10567647 podStartE2EDuration="6.670616131s" totalImagesPullingTime="4.564939661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:07:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:07:49.669139283 +0000 UTC m=+380.221032007" watchObservedRunningTime="2026-07-20 13:07:49.670616131 +0000 UTC m=+380.222508829" Jul 20 13:07:50.568203 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:50.568176 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:51.024821 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.024786 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84676b455f-njj7p"] Jul 20 13:07:51.028276 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.028258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.032210 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.032181 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 13:07:51.032339 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.032226 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 13:07:51.032339 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.032244 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 13:07:51.032339 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.032195 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g6p65\"" Jul 20 13:07:51.032339 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.032276 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 13:07:51.032590 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.032573 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 13:07:51.038813 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.038793 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84676b455f-njj7p"] Jul 20 13:07:51.142547 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.142510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-metrics-cert\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.142694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.142565 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-cert\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.142694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.142626 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-manager-config\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.142764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.142701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4kkx\" (UniqueName: \"kubernetes.io/projected/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-kube-api-access-s4kkx\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.244170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.244137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4kkx\" (UniqueName: \"kubernetes.io/projected/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-kube-api-access-s4kkx\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.244325 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.244177 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-metrics-cert\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.244379 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.244358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-cert\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.244462 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.244442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-manager-config\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.245052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.245028 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-manager-config\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.246588 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.246563 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-metrics-cert\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.246828 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.246808 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-cert\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.264235 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.264212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4kkx\" (UniqueName: \"kubernetes.io/projected/f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b-kube-api-access-s4kkx\") pod \"lws-controller-manager-84676b455f-njj7p\" (UID: \"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b\") " pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.337383 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.337316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:51.517242 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.517210 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84676b455f-njj7p"] Jul 20 13:07:51.520060 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:07:51.520034 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ccd0f0_eb96_4aaa_b0e3_ac1a8b7df66b.slice/crio-8fe58d82809fa8908afd0fd830731c0148c799eb9658e889e5a052c3dcfd54a7 WatchSource:0}: Error finding container 8fe58d82809fa8908afd0fd830731c0148c799eb9658e889e5a052c3dcfd54a7: Status 404 returned error can't find the container with id 8fe58d82809fa8908afd0fd830731c0148c799eb9658e889e5a052c3dcfd54a7 Jul 20 13:07:51.571945 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:51.571916 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" event={"ID":"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b","Type":"ContainerStarted","Data":"8fe58d82809fa8908afd0fd830731c0148c799eb9658e889e5a052c3dcfd54a7"} Jul 20 13:07:54.763634 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:54.763598 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb"] Jul 20 13:07:54.764153 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:54.763800 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" podUID="a4fab8b8-969d-4594-a8c9-5571a74558e4" containerName="discovery" containerID="cri-o://0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e" gracePeriod=30 Jul 20 13:07:55.005726 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.005700 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:55.078752 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078676 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fab8b8-969d-4594-a8c9-5571a74558e4-local-certs\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.078752 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078710 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-cacerts\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.078752 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078737 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-kubeconfig\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.079052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078757 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-dns-cert\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.079052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078787 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-token\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.079052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078824 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-ca-configmap\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.079052 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.078854 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg2hp\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-kube-api-access-lg2hp\") pod \"a4fab8b8-969d-4594-a8c9-5571a74558e4\" (UID: \"a4fab8b8-969d-4594-a8c9-5571a74558e4\") " Jul 20 13:07:55.079333 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.079295 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 13:07:55.081229 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.081201 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:07:55.081334 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.081259 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:07:55.081415 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.081390 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fab8b8-969d-4594-a8c9-5571a74558e4-local-certs" (OuterVolumeSpecName: "local-certs") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:07:55.081510 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.081465 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-kube-api-access-lg2hp" (OuterVolumeSpecName: "kube-api-access-lg2hp") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "kube-api-access-lg2hp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:07:55.081510 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.081497 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:07:55.081617 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.081603 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-cacerts" (OuterVolumeSpecName: "cacerts") pod "a4fab8b8-969d-4594-a8c9-5571a74558e4" (UID: "a4fab8b8-969d-4594-a8c9-5571a74558e4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:07:55.180036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180011 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fab8b8-969d-4594-a8c9-5571a74558e4-local-certs\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.180036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180033 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-cacerts\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.180036 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180042 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-kubeconfig\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.180236 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180052 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-dns-cert\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.180236 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180062 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-token\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.180236 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180071 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fab8b8-969d-4594-a8c9-5571a74558e4-istio-csr-ca-configmap\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.180236 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.180081 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg2hp\" (UniqueName: \"kubernetes.io/projected/a4fab8b8-969d-4594-a8c9-5571a74558e4-kube-api-access-lg2hp\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:07:55.590715 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.590680 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4fab8b8-969d-4594-a8c9-5571a74558e4" containerID="0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e" exitCode=0 Jul 20 13:07:55.590896 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.590718 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" Jul 20 13:07:55.590896 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.590767 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" event={"ID":"a4fab8b8-969d-4594-a8c9-5571a74558e4","Type":"ContainerDied","Data":"0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e"} Jul 20 13:07:55.590896 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.590820 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb" event={"ID":"a4fab8b8-969d-4594-a8c9-5571a74558e4","Type":"ContainerDied","Data":"ff0e9b5a816c737bc973cf0f41f5b874cb2557386a8b57a07fa709268ac9503c"} Jul 20 13:07:55.590896 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.590841 2564 scope.go:117] "RemoveContainer" containerID="0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e" Jul 20 13:07:55.599990 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.599972 2564 scope.go:117] "RemoveContainer" containerID="0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e" Jul 20 13:07:55.600281 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:07:55.600252 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e\": container with ID starting with 0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e not found: ID does not exist" containerID="0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e" Jul 20 13:07:55.600383 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.600290 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e"} err="failed to get container status \"0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e\": rpc error: code = NotFound desc = could not find container \"0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e\": container with ID starting with 0efccf6f4e2a85e04e905b8a190b6380338687ca5f5c8025d5d0231d139c1d1e not found: ID does not exist" Jul 20 13:07:55.614511 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.614484 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb"] Jul 20 13:07:55.619334 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.619310 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hvmvb"] Jul 20 13:07:55.937661 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:55.937634 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4fab8b8-969d-4594-a8c9-5571a74558e4" path="/var/lib/kubelet/pods/a4fab8b8-969d-4594-a8c9-5571a74558e4/volumes" Jul 20 13:07:57.602241 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:57.602205 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" event={"ID":"f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b","Type":"ContainerStarted","Data":"c01d0cab63a3145052b6eef572f22b2768bfff4d2551815010db0f0cfb9f438c"} Jul 20 13:07:57.602665 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:57.602341 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:07:57.621101 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:07:57.621056 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" podStartSLOduration=1.312198298 podStartE2EDuration="6.621042201s" totalImagesPullingTime="5.308843903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:07:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:07:57.620101789 +0000 UTC m=+388.171994510" watchObservedRunningTime="2026-07-20 13:07:57.621042201 +0000 UTC m=+388.172934901" Jul 20 13:08:00.568532 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:08:00.568502 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2xtgl" Jul 20 13:08:00.570666 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:08:00.570643 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jq7bz" Jul 20 13:08:08.609964 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:08:08.609930 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84676b455f-njj7p" Jul 20 13:09:43.995140 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:43.995101 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh"] Jul 20 13:09:43.995608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:43.995404 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4fab8b8-969d-4594-a8c9-5571a74558e4" containerName="discovery" Jul 20 13:09:43.995608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:43.995415 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fab8b8-969d-4594-a8c9-5571a74558e4" containerName="discovery" Jul 20 13:09:43.995608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:43.995472 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4fab8b8-969d-4594-a8c9-5571a74558e4" containerName="discovery" Jul 20 13:09:43.998305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:43.998285 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.001915 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.001894 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 13:09:44.002983 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.002968 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 13:09:44.003154 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.003136 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 13:09:44.003214 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.003193 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 13:09:44.003263 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.003215 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kpwv5\"" Jul 20 13:09:44.020665 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.020638 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh"] Jul 20 13:09:44.033311 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.033281 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bd7\" (UniqueName: \"kubernetes.io/projected/d440c203-acea-433e-8f98-dbc53192af42-kube-api-access-p9bd7\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.033431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.033365 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d440c203-acea-433e-8f98-dbc53192af42-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.033431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.033411 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d440c203-acea-433e-8f98-dbc53192af42-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.134202 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.134169 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d440c203-acea-433e-8f98-dbc53192af42-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.134374 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.134224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd7\" (UniqueName: \"kubernetes.io/projected/d440c203-acea-433e-8f98-dbc53192af42-kube-api-access-p9bd7\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.134374 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.134267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d440c203-acea-433e-8f98-dbc53192af42-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.134374 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:09:44.134363 2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 20 13:09:44.134553 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:09:44.134428 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d440c203-acea-433e-8f98-dbc53192af42-plugin-serving-cert podName:d440c203-acea-433e-8f98-dbc53192af42 nodeName:}" failed. No retries permitted until 2026-07-20 13:09:44.634407869 +0000 UTC m=+495.186300545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/d440c203-acea-433e-8f98-dbc53192af42-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-dvvwh" (UID: "d440c203-acea-433e-8f98-dbc53192af42") : secret "plugin-serving-cert" not found Jul 20 13:09:44.134870 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.134846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d440c203-acea-433e-8f98-dbc53192af42-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.143203 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.143173 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bd7\" (UniqueName: \"kubernetes.io/projected/d440c203-acea-433e-8f98-dbc53192af42-kube-api-access-p9bd7\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.636471 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.636421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d440c203-acea-433e-8f98-dbc53192af42-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.638862 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.638840 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d440c203-acea-433e-8f98-dbc53192af42-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dvvwh\" (UID: \"d440c203-acea-433e-8f98-dbc53192af42\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:44.907468 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:44.907384 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" Jul 20 13:09:45.052554 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:45.052529 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh"] Jul 20 13:09:45.055357 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:09:45.055330 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd440c203_acea_433e_8f98_dbc53192af42.slice/crio-c380476a5a1893bb82ae7925710a251bda8ed098db5cd46e7e4c6ec03ed33a2e WatchSource:0}: Error finding container c380476a5a1893bb82ae7925710a251bda8ed098db5cd46e7e4c6ec03ed33a2e: Status 404 returned error can't find the container with id c380476a5a1893bb82ae7925710a251bda8ed098db5cd46e7e4c6ec03ed33a2e Jul 20 13:09:45.076171 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:09:45.076140 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" event={"ID":"d440c203-acea-433e-8f98-dbc53192af42","Type":"ContainerStarted","Data":"c380476a5a1893bb82ae7925710a251bda8ed098db5cd46e7e4c6ec03ed33a2e"} Jul 20 13:10:07.194893 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:07.194841 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" event={"ID":"d440c203-acea-433e-8f98-dbc53192af42","Type":"ContainerStarted","Data":"7abdf38464a7e7ab9e0f6aefbe12e566ea71f0a8b653f87954de2133e794461d"} Jul 20 13:10:07.220168 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:07.220115 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dvvwh" podStartSLOduration=2.493610437 podStartE2EDuration="24.220098876s" totalImagesPullingTime="21.726488439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:10:07.218696334 +0000 UTC m=+517.770589033" watchObservedRunningTime="2026-07-20 13:10:07.220098876 +0000 UTC m=+517.771991574" Jul 20 13:10:41.743703 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.743665 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6lvgr"] Jul 20 13:10:41.767973 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.767937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lvgr"] Jul 20 13:10:41.768120 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.768047 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:41.773103 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.773083 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cfngg\"" Jul 20 13:10:41.832695 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.832664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgk2p\" (UniqueName: \"kubernetes.io/projected/63fe2ea1-e402-4f33-9bea-692c0f93b7ba-kube-api-access-vgk2p\") pod \"authorino-58cd955675-6lvgr\" (UID: \"63fe2ea1-e402-4f33-9bea-692c0f93b7ba\") " pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:41.933199 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.933166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgk2p\" (UniqueName: \"kubernetes.io/projected/63fe2ea1-e402-4f33-9bea-692c0f93b7ba-kube-api-access-vgk2p\") pod \"authorino-58cd955675-6lvgr\" (UID: \"63fe2ea1-e402-4f33-9bea-692c0f93b7ba\") " pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:41.943954 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.943930 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgk2p\" (UniqueName: \"kubernetes.io/projected/63fe2ea1-e402-4f33-9bea-692c0f93b7ba-kube-api-access-vgk2p\") pod \"authorino-58cd955675-6lvgr\" (UID: \"63fe2ea1-e402-4f33-9bea-692c0f93b7ba\") " pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:41.986678 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.986654 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hhg59"] Jul 20 13:10:41.989469 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:41.989451 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:10:42.005271 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.005246 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hhg59"] Jul 20 13:10:42.033571 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.033544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2947n\" (UniqueName: \"kubernetes.io/projected/d388e86a-b9d5-4778-84a1-5620e4e7b12b-kube-api-access-2947n\") pod \"authorino-75d556bd77-hhg59\" (UID: \"d388e86a-b9d5-4778-84a1-5620e4e7b12b\") " pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:10:42.076977 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.076954 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:42.134643 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.134596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2947n\" (UniqueName: \"kubernetes.io/projected/d388e86a-b9d5-4778-84a1-5620e4e7b12b-kube-api-access-2947n\") pod \"authorino-75d556bd77-hhg59\" (UID: \"d388e86a-b9d5-4778-84a1-5620e4e7b12b\") " pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:10:42.144845 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.144814 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2947n\" (UniqueName: \"kubernetes.io/projected/d388e86a-b9d5-4778-84a1-5620e4e7b12b-kube-api-access-2947n\") pod \"authorino-75d556bd77-hhg59\" (UID: \"d388e86a-b9d5-4778-84a1-5620e4e7b12b\") " pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:10:42.206582 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.206531 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lvgr"] Jul 20 13:10:42.208652 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:10:42.208626 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63fe2ea1_e402_4f33_9bea_692c0f93b7ba.slice/crio-a6d23be8b1b9636f5a49c938376012255557553b8cb6745dfcbd0363a773664b WatchSource:0}: Error finding container a6d23be8b1b9636f5a49c938376012255557553b8cb6745dfcbd0363a773664b: Status 404 returned error can't find the container with id a6d23be8b1b9636f5a49c938376012255557553b8cb6745dfcbd0363a773664b Jul 20 13:10:42.298097 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.298033 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:10:42.353311 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.353270 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lvgr" event={"ID":"63fe2ea1-e402-4f33-9bea-692c0f93b7ba","Type":"ContainerStarted","Data":"a6d23be8b1b9636f5a49c938376012255557553b8cb6745dfcbd0363a773664b"} Jul 20 13:10:42.426574 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:42.426549 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hhg59"] Jul 20 13:10:42.428746 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:10:42.428721 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd388e86a_b9d5_4778_84a1_5620e4e7b12b.slice/crio-c05213ae229f660e008ed239babcfe91ef6995f5db5d87b0669f9eba066a700e WatchSource:0}: Error finding container c05213ae229f660e008ed239babcfe91ef6995f5db5d87b0669f9eba066a700e: Status 404 returned error can't find the container with id c05213ae229f660e008ed239babcfe91ef6995f5db5d87b0669f9eba066a700e Jul 20 13:10:43.361064 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:43.361025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hhg59" event={"ID":"d388e86a-b9d5-4778-84a1-5620e4e7b12b","Type":"ContainerStarted","Data":"c05213ae229f660e008ed239babcfe91ef6995f5db5d87b0669f9eba066a700e"} Jul 20 13:10:46.381234 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:46.381135 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lvgr" event={"ID":"63fe2ea1-e402-4f33-9bea-692c0f93b7ba","Type":"ContainerStarted","Data":"e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2"} Jul 20 13:10:46.382547 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:46.382525 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hhg59" event={"ID":"d388e86a-b9d5-4778-84a1-5620e4e7b12b","Type":"ContainerStarted","Data":"f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819"} Jul 20 13:10:46.399611 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:46.399565 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6lvgr" podStartSLOduration=1.6401901589999999 podStartE2EDuration="5.399551541s" totalImagesPullingTime="3.759361382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:10:46.397741862 +0000 UTC m=+556.949634562" watchObservedRunningTime="2026-07-20 13:10:46.399551541 +0000 UTC m=+556.951444269" Jul 20 13:10:46.421156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:46.421107 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hhg59" podStartSLOduration=1.895665403 podStartE2EDuration="5.421092986s" totalImagesPullingTime="3.525427583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:10:46.419492631 +0000 UTC m=+556.971385331" watchObservedRunningTime="2026-07-20 13:10:46.421092986 +0000 UTC m=+556.972985690" Jul 20 13:10:46.442263 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:46.442232 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lvgr"] Jul 20 13:10:48.392286 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:48.392227 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6lvgr" podUID="63fe2ea1-e402-4f33-9bea-692c0f93b7ba" containerName="authorino" containerID="cri-o://e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2" gracePeriod=30 Jul 20 13:10:48.637221 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:48.637197 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:48.686810 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:48.686788 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgk2p\" (UniqueName: \"kubernetes.io/projected/63fe2ea1-e402-4f33-9bea-692c0f93b7ba-kube-api-access-vgk2p\") pod \"63fe2ea1-e402-4f33-9bea-692c0f93b7ba\" (UID: \"63fe2ea1-e402-4f33-9bea-692c0f93b7ba\") " Jul 20 13:10:48.688902 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:48.688864 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63fe2ea1-e402-4f33-9bea-692c0f93b7ba-kube-api-access-vgk2p" (OuterVolumeSpecName: "kube-api-access-vgk2p") pod "63fe2ea1-e402-4f33-9bea-692c0f93b7ba" (UID: "63fe2ea1-e402-4f33-9bea-692c0f93b7ba"). InnerVolumeSpecName "kube-api-access-vgk2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:10:48.788148 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:48.788104 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgk2p\" (UniqueName: \"kubernetes.io/projected/63fe2ea1-e402-4f33-9bea-692c0f93b7ba-kube-api-access-vgk2p\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:10:49.397985 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.397949 2564 generic.go:356] "Generic (PLEG): container finished" podID="63fe2ea1-e402-4f33-9bea-692c0f93b7ba" containerID="e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2" exitCode=0 Jul 20 13:10:49.397985 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.397986 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lvgr" Jul 20 13:10:49.398494 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.398037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lvgr" event={"ID":"63fe2ea1-e402-4f33-9bea-692c0f93b7ba","Type":"ContainerDied","Data":"e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2"} Jul 20 13:10:49.398494 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.398073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lvgr" event={"ID":"63fe2ea1-e402-4f33-9bea-692c0f93b7ba","Type":"ContainerDied","Data":"a6d23be8b1b9636f5a49c938376012255557553b8cb6745dfcbd0363a773664b"} Jul 20 13:10:49.398494 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.398089 2564 scope.go:117] "RemoveContainer" containerID="e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2" Jul 20 13:10:49.406460 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.406444 2564 scope.go:117] "RemoveContainer" containerID="e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2" Jul 20 13:10:49.406712 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:10:49.406692 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2\": container with ID starting with e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2 not found: ID does not exist" containerID="e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2" Jul 20 13:10:49.406775 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.406724 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2"} err="failed to get container status \"e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2\": rpc error: code = NotFound desc = could not find container \"e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2\": container with ID starting with e434335e8deadfb6d87d873ff4482e38e7cf8c5ab22ce18c9ab578308f1c79a2 not found: ID does not exist" Jul 20 13:10:49.421967 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.421942 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lvgr"] Jul 20 13:10:49.425491 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.425469 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lvgr"] Jul 20 13:10:49.935285 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:10:49.935255 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63fe2ea1-e402-4f33-9bea-692c0f93b7ba" path="/var/lib/kubelet/pods/63fe2ea1-e402-4f33-9bea-692c0f93b7ba/volumes" Jul 20 13:11:09.764019 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.763984 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f74585759-8qgkj"] Jul 20 13:11:09.764462 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.764299 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63fe2ea1-e402-4f33-9bea-692c0f93b7ba" containerName="authorino" Jul 20 13:11:09.764462 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.764312 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63fe2ea1-e402-4f33-9bea-692c0f93b7ba" containerName="authorino" Jul 20 13:11:09.764462 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.764381 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63fe2ea1-e402-4f33-9bea-692c0f93b7ba" containerName="authorino" Jul 20 13:11:09.767389 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.767368 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:09.769793 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.769773 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 20 13:11:09.774679 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.774661 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f74585759-8qgkj"] Jul 20 13:11:09.863826 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.863791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-tls-cert\") pod \"authorino-7f74585759-8qgkj\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:09.863826 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.863840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdwwq\" (UniqueName: \"kubernetes.io/projected/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-kube-api-access-bdwwq\") pod \"authorino-7f74585759-8qgkj\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:09.965018 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.964987 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-tls-cert\") pod \"authorino-7f74585759-8qgkj\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:09.965183 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.965029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdwwq\" (UniqueName: \"kubernetes.io/projected/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-kube-api-access-bdwwq\") pod \"authorino-7f74585759-8qgkj\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:09.967427 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.967398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-tls-cert\") pod \"authorino-7f74585759-8qgkj\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:09.973611 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:09.973579 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdwwq\" (UniqueName: \"kubernetes.io/projected/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-kube-api-access-bdwwq\") pod \"authorino-7f74585759-8qgkj\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:10.077281 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:10.077191 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:11:10.206397 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:10.206373 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f74585759-8qgkj"] Jul 20 13:11:10.208476 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:11:10.208448 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e5f5199_36a9_4c52_a91e_7a492a6f3e2f.slice/crio-d7327b525855eaacf9bf520b8d497972f8123afe1375aa757b7d6478f3ddc36f WatchSource:0}: Error finding container d7327b525855eaacf9bf520b8d497972f8123afe1375aa757b7d6478f3ddc36f: Status 404 returned error can't find the container with id d7327b525855eaacf9bf520b8d497972f8123afe1375aa757b7d6478f3ddc36f Jul 20 13:11:10.492240 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:10.492208 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f74585759-8qgkj" event={"ID":"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f","Type":"ContainerStarted","Data":"d7327b525855eaacf9bf520b8d497972f8123afe1375aa757b7d6478f3ddc36f"} Jul 20 13:11:11.498140 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.498103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f74585759-8qgkj" event={"ID":"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f","Type":"ContainerStarted","Data":"c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7"} Jul 20 13:11:11.542430 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.542378 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f74585759-8qgkj" podStartSLOduration=2.197245042 podStartE2EDuration="2.542362925s" totalImagesPullingTime="345.117883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:11:11.515961469 +0000 UTC m=+582.067854169" watchObservedRunningTime="2026-07-20 13:11:11.542362925 +0000 UTC m=+582.094255623" Jul 20 13:11:11.543820 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.543797 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hhg59"] Jul 20 13:11:11.544032 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.544009 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hhg59" podUID="d388e86a-b9d5-4778-84a1-5620e4e7b12b" containerName="authorino" containerID="cri-o://f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819" gracePeriod=30 Jul 20 13:11:11.793349 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.793325 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:11:11.881515 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.881470 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2947n\" (UniqueName: \"kubernetes.io/projected/d388e86a-b9d5-4778-84a1-5620e4e7b12b-kube-api-access-2947n\") pod \"d388e86a-b9d5-4778-84a1-5620e4e7b12b\" (UID: \"d388e86a-b9d5-4778-84a1-5620e4e7b12b\") " Jul 20 13:11:11.883480 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.883452 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d388e86a-b9d5-4778-84a1-5620e4e7b12b-kube-api-access-2947n" (OuterVolumeSpecName: "kube-api-access-2947n") pod "d388e86a-b9d5-4778-84a1-5620e4e7b12b" (UID: "d388e86a-b9d5-4778-84a1-5620e4e7b12b"). InnerVolumeSpecName "kube-api-access-2947n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:11:11.982917 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:11.982867 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2947n\" (UniqueName: \"kubernetes.io/projected/d388e86a-b9d5-4778-84a1-5620e4e7b12b-kube-api-access-2947n\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:11:12.503971 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.503935 2564 generic.go:356] "Generic (PLEG): container finished" podID="d388e86a-b9d5-4778-84a1-5620e4e7b12b" containerID="f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819" exitCode=0 Jul 20 13:11:12.504385 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.503977 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hhg59" Jul 20 13:11:12.504385 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.504020 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hhg59" event={"ID":"d388e86a-b9d5-4778-84a1-5620e4e7b12b","Type":"ContainerDied","Data":"f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819"} Jul 20 13:11:12.504385 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.504061 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hhg59" event={"ID":"d388e86a-b9d5-4778-84a1-5620e4e7b12b","Type":"ContainerDied","Data":"c05213ae229f660e008ed239babcfe91ef6995f5db5d87b0669f9eba066a700e"} Jul 20 13:11:12.504385 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.504078 2564 scope.go:117] "RemoveContainer" containerID="f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819" Jul 20 13:11:12.512529 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.512509 2564 scope.go:117] "RemoveContainer" containerID="f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819" Jul 20 13:11:12.512815 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:11:12.512789 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819\": container with ID starting with f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819 not found: ID does not exist" containerID="f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819" Jul 20 13:11:12.512902 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.512822 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819"} err="failed to get container status \"f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819\": rpc error: code = NotFound desc = could not find container \"f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819\": container with ID starting with f347f927e6f3195a8d904b1d25539e97c99aac1ca59fa85da04a980511154819 not found: ID does not exist" Jul 20 13:11:12.519908 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.519863 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hhg59"] Jul 20 13:11:12.523557 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:12.523538 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hhg59"] Jul 20 13:11:13.935357 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:13.935327 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d388e86a-b9d5-4778-84a1-5620e4e7b12b" path="/var/lib/kubelet/pods/d388e86a-b9d5-4778-84a1-5620e4e7b12b/volumes" Jul 20 13:11:29.836362 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:29.836331 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:11:29.839417 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:29.839395 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:11:29.840346 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:29.840325 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:11:29.843421 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:11:29.843393 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:12:35.019058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.019017 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps"] Jul 20 13:12:35.019631 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.019335 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d388e86a-b9d5-4778-84a1-5620e4e7b12b" containerName="authorino" Jul 20 13:12:35.019631 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.019347 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d388e86a-b9d5-4778-84a1-5620e4e7b12b" containerName="authorino" Jul 20 13:12:35.019631 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.019422 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d388e86a-b9d5-4778-84a1-5620e4e7b12b" containerName="authorino" Jul 20 13:12:35.022314 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.022289 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.026183 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.026163 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 13:12:35.026305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.026163 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x7r74\"" Jul 20 13:12:35.026305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.026165 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 20 13:12:35.026305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.026241 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 13:12:35.034133 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.034112 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps"] Jul 20 13:12:35.090511 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.090477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9hqh\" (UniqueName: \"kubernetes.io/projected/51e9f150-09e9-4a04-8bc1-7b362155b57d-kube-api-access-l9hqh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.090676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.090538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.090676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.090560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.090676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.090581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.090676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.090620 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.090676 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.090651 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51e9f150-09e9-4a04-8bc1-7b362155b57d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191218 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191218 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191219 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51e9f150-09e9-4a04-8bc1-7b362155b57d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191472 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9hqh\" (UniqueName: \"kubernetes.io/projected/51e9f150-09e9-4a04-8bc1-7b362155b57d-kube-api-access-l9hqh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191472 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191472 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191472 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191697 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191673 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.191783 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.191742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.193608 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.193586 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51e9f150-09e9-4a04-8bc1-7b362155b57d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.193858 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.193841 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51e9f150-09e9-4a04-8bc1-7b362155b57d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.201633 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.201606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9hqh\" (UniqueName: \"kubernetes.io/projected/51e9f150-09e9-4a04-8bc1-7b362155b57d-kube-api-access-l9hqh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-m74ps\" (UID: \"51e9f150-09e9-4a04-8bc1-7b362155b57d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.333950 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.333828 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:35.466701 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.466676 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps"] Jul 20 13:12:35.468108 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:12:35.468084 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e9f150_09e9_4a04_8bc1_7b362155b57d.slice/crio-e563edbb9ece61d9e138168a18ceda29b7f731f9f1067fcbda09bdadc881f83c WatchSource:0}: Error finding container e563edbb9ece61d9e138168a18ceda29b7f731f9f1067fcbda09bdadc881f83c: Status 404 returned error can't find the container with id e563edbb9ece61d9e138168a18ceda29b7f731f9f1067fcbda09bdadc881f83c Jul 20 13:12:35.469386 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.469365 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 13:12:35.886733 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:35.886688 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" event={"ID":"51e9f150-09e9-4a04-8bc1-7b362155b57d","Type":"ContainerStarted","Data":"e563edbb9ece61d9e138168a18ceda29b7f731f9f1067fcbda09bdadc881f83c"} Jul 20 13:12:36.894622 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:36.894515 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" event={"ID":"51e9f150-09e9-4a04-8bc1-7b362155b57d","Type":"ContainerStarted","Data":"53d0b11ba4d112781c69e8b323d51a7dceeed7f5e9eab41410dfa09532b08896"} Jul 20 13:12:36.915406 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:36.915344 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" podStartSLOduration=1.808554199 podStartE2EDuration="2.915326984s" totalImagesPullingTime="1.106772785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:12:36.914055821 +0000 UTC m=+667.465948562" watchObservedRunningTime="2026-07-20 13:12:36.915326984 +0000 UTC m=+667.467219702" Jul 20 13:12:45.334466 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:45.334429 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:45.334903 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:45.334698 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:45.347208 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:45.347183 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:45.951453 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:45.951418 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-m74ps" Jul 20 13:12:47.817747 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.817714 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822"] Jul 20 13:12:47.821031 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.821014 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:47.823601 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.823577 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 20 13:12:47.832316 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.832292 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822"] Jul 20 13:12:47.902478 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.902446 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:47.902618 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.902494 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv7s8\" (UniqueName: \"kubernetes.io/projected/d760ff6e-2e59-4966-af23-be88740bc841-kube-api-access-nv7s8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:47.902618 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.902589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:47.902694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.902626 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:47.902694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.902646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:47.902694 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:47.902678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d760ff6e-2e59-4966-af23-be88740bc841-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.003646 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.003615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.003803 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.003651 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.003803 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.003669 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.003803 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.003684 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d760ff6e-2e59-4966-af23-be88740bc841-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.003803 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.003757 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.004058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.003829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv7s8\" (UniqueName: \"kubernetes.io/projected/d760ff6e-2e59-4966-af23-be88740bc841-kube-api-access-nv7s8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.004114 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.004060 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.004184 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.004160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.004241 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.004186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.006037 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.006011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d760ff6e-2e59-4966-af23-be88740bc841-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.006465 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.006449 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d760ff6e-2e59-4966-af23-be88740bc841-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.012944 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.012905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv7s8\" (UniqueName: \"kubernetes.io/projected/d760ff6e-2e59-4966-af23-be88740bc841-kube-api-access-nv7s8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xd822\" (UID: \"d760ff6e-2e59-4966-af23-be88740bc841\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.130656 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.130573 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:48.276020 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.275996 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822"] Jul 20 13:12:48.278130 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:12:48.278097 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd760ff6e_2e59_4966_af23_be88740bc841.slice/crio-b7b85526174f6ec740f8a13a163d617855515a7601e4474b5f0dfa5d500f1d74 WatchSource:0}: Error finding container b7b85526174f6ec740f8a13a163d617855515a7601e4474b5f0dfa5d500f1d74: Status 404 returned error can't find the container with id b7b85526174f6ec740f8a13a163d617855515a7601e4474b5f0dfa5d500f1d74 Jul 20 13:12:48.952855 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.952817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" event={"ID":"d760ff6e-2e59-4966-af23-be88740bc841","Type":"ContainerStarted","Data":"5fe6a1058196dfbf473fddd09dd0a8e69d89c488ca623aa32e66ea5af1605606"} Jul 20 13:12:48.953269 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:48.952897 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" event={"ID":"d760ff6e-2e59-4966-af23-be88740bc841","Type":"ContainerStarted","Data":"b7b85526174f6ec740f8a13a163d617855515a7601e4474b5f0dfa5d500f1d74"} Jul 20 13:12:49.006092 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:49.006043 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" podStartSLOduration=1.7353764109999998 podStartE2EDuration="2.006028045s" totalImagesPullingTime="270.651634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:12:48.994219546 +0000 UTC m=+679.546112243" watchObservedRunningTime="2026-07-20 13:12:49.006028045 +0000 UTC m=+679.557920742" Jul 20 13:12:58.131652 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:58.131623 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:58.132065 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:58.131698 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:58.144182 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:58.144159 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:12:59.011782 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:12:59.011755 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xd822" Jul 20 13:13:00.743815 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.743783 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr"] Jul 20 13:13:00.746962 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.746941 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:00.750301 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.750283 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 20 13:13:00.758960 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.758936 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr"] Jul 20 13:13:00.914986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.914945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:00.914986 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.914990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:00.915184 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.915008 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:00.915184 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.915027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zxm6\" (UniqueName: \"kubernetes.io/projected/08c98ed7-3359-4c23-9433-7abcd40cd16e-kube-api-access-6zxm6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:00.915184 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.915092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:00.915184 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:00.915139 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08c98ed7-3359-4c23-9433-7abcd40cd16e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.015583 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.015583 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.015583 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.015834 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015676 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zxm6\" (UniqueName: \"kubernetes.io/projected/08c98ed7-3359-4c23-9433-7abcd40cd16e-kube-api-access-6zxm6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.015834 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.015834 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08c98ed7-3359-4c23-9433-7abcd40cd16e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.016034 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.016034 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.015964 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.016117 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.016031 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.018005 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.017982 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08c98ed7-3359-4c23-9433-7abcd40cd16e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.018117 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.018100 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08c98ed7-3359-4c23-9433-7abcd40cd16e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.024702 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.024680 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zxm6\" (UniqueName: \"kubernetes.io/projected/08c98ed7-3359-4c23-9433-7abcd40cd16e-kube-api-access-6zxm6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr\" (UID: \"08c98ed7-3359-4c23-9433-7abcd40cd16e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.056633 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.056610 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:01.189333 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:01.189310 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr"] Jul 20 13:13:01.191145 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:13:01.191116 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08c98ed7_3359_4c23_9433_7abcd40cd16e.slice/crio-77151fbd8ecb07f93a147cdc3ac192530c1d6e0fb91aab89058a3e11655ff0c9 WatchSource:0}: Error finding container 77151fbd8ecb07f93a147cdc3ac192530c1d6e0fb91aab89058a3e11655ff0c9: Status 404 returned error can't find the container with id 77151fbd8ecb07f93a147cdc3ac192530c1d6e0fb91aab89058a3e11655ff0c9 Jul 20 13:13:02.014979 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:02.014943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" event={"ID":"08c98ed7-3359-4c23-9433-7abcd40cd16e","Type":"ContainerStarted","Data":"22b7bd1e88dfa4a915813bd1dbe06dabbeb8c19c87f87c329a1cb60930e0c39d"} Jul 20 13:13:02.015366 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:02.014986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" event={"ID":"08c98ed7-3359-4c23-9433-7abcd40cd16e","Type":"ContainerStarted","Data":"77151fbd8ecb07f93a147cdc3ac192530c1d6e0fb91aab89058a3e11655ff0c9"} Jul 20 13:13:02.035428 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:02.035366 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" podStartSLOduration=1.7983592929999999 podStartE2EDuration="2.035350747s" totalImagesPullingTime="236.991454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:13:02.034843083 +0000 UTC m=+692.586735783" watchObservedRunningTime="2026-07-20 13:13:02.035350747 +0000 UTC m=+692.587243471" Jul 20 13:13:11.057368 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:11.057273 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:11.057368 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:11.057335 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:11.070467 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:11.070440 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:13:12.076821 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:13:12.076769 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr" Jul 20 13:14:28.366368 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.366331 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d755cbb74-4ghj5"] Jul 20 13:14:28.368909 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.368871 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.377820 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.377797 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d755cbb74-4ghj5"] Jul 20 13:14:28.451720 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.451692 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d81ca6d-32e6-4128-adfe-49c890a6028f-tls-cert\") pod \"authorino-d755cbb74-4ghj5\" (UID: \"4d81ca6d-32e6-4128-adfe-49c890a6028f\") " pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.451865 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.451740 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74k5m\" (UniqueName: \"kubernetes.io/projected/4d81ca6d-32e6-4128-adfe-49c890a6028f-kube-api-access-74k5m\") pod \"authorino-d755cbb74-4ghj5\" (UID: \"4d81ca6d-32e6-4128-adfe-49c890a6028f\") " pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.553115 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.553079 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d81ca6d-32e6-4128-adfe-49c890a6028f-tls-cert\") pod \"authorino-d755cbb74-4ghj5\" (UID: \"4d81ca6d-32e6-4128-adfe-49c890a6028f\") " pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.553269 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.553131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74k5m\" (UniqueName: \"kubernetes.io/projected/4d81ca6d-32e6-4128-adfe-49c890a6028f-kube-api-access-74k5m\") pod \"authorino-d755cbb74-4ghj5\" (UID: \"4d81ca6d-32e6-4128-adfe-49c890a6028f\") " pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.555540 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.555512 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d81ca6d-32e6-4128-adfe-49c890a6028f-tls-cert\") pod \"authorino-d755cbb74-4ghj5\" (UID: \"4d81ca6d-32e6-4128-adfe-49c890a6028f\") " pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.562967 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.562945 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74k5m\" (UniqueName: \"kubernetes.io/projected/4d81ca6d-32e6-4128-adfe-49c890a6028f-kube-api-access-74k5m\") pod \"authorino-d755cbb74-4ghj5\" (UID: \"4d81ca6d-32e6-4128-adfe-49c890a6028f\") " pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.678415 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.678344 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d755cbb74-4ghj5" Jul 20 13:14:28.825858 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:28.825823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d755cbb74-4ghj5"] Jul 20 13:14:28.829367 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:14:28.829335 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d81ca6d_32e6_4128_adfe_49c890a6028f.slice/crio-5286738b063e5611972288c7cdd82f52c4470e068a1d6184b865090ecedf9743 WatchSource:0}: Error finding container 5286738b063e5611972288c7cdd82f52c4470e068a1d6184b865090ecedf9743: Status 404 returned error can't find the container with id 5286738b063e5611972288c7cdd82f52c4470e068a1d6184b865090ecedf9743 Jul 20 13:14:29.412682 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:29.412644 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d755cbb74-4ghj5" event={"ID":"4d81ca6d-32e6-4128-adfe-49c890a6028f","Type":"ContainerStarted","Data":"5286738b063e5611972288c7cdd82f52c4470e068a1d6184b865090ecedf9743"} Jul 20 13:14:30.419622 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.419570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d755cbb74-4ghj5" event={"ID":"4d81ca6d-32e6-4128-adfe-49c890a6028f","Type":"ContainerStarted","Data":"d3b3741692900e236856fbb3ffdd923e3f093f0e43ad8d790525677c339ffbd7"} Jul 20 13:14:30.447938 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.447753 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d755cbb74-4ghj5" podStartSLOduration=1.8012196980000001 podStartE2EDuration="2.44772846s" totalImagesPullingTime="646.508762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:14:30.444871446 +0000 UTC m=+780.996764146" watchObservedRunningTime="2026-07-20 13:14:30.44772846 +0000 UTC m=+780.999621159" Jul 20 13:14:30.496938 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.496905 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f74585759-8qgkj"] Jul 20 13:14:30.497134 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.497097 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7f74585759-8qgkj" podUID="6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" containerName="authorino" containerID="cri-o://c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7" gracePeriod=30 Jul 20 13:14:30.768542 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.768507 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:14:30.876374 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.876336 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-tls-cert\") pod \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " Jul 20 13:14:30.876543 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.876394 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdwwq\" (UniqueName: \"kubernetes.io/projected/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-kube-api-access-bdwwq\") pod \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\" (UID: \"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f\") " Jul 20 13:14:30.878495 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.878466 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-kube-api-access-bdwwq" (OuterVolumeSpecName: "kube-api-access-bdwwq") pod "6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" (UID: "6e5f5199-36a9-4c52-a91e-7a492a6f3e2f"). InnerVolumeSpecName "kube-api-access-bdwwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:14:30.888230 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.888193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" (UID: "6e5f5199-36a9-4c52-a91e-7a492a6f3e2f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:14:30.977150 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.977112 2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-tls-cert\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:14:30.977150 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:30.977145 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdwwq\" (UniqueName: \"kubernetes.io/projected/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f-kube-api-access-bdwwq\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:14:31.425554 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.425466 2564 generic.go:356] "Generic (PLEG): container finished" podID="6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" containerID="c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7" exitCode=0 Jul 20 13:14:31.425554 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.425500 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f74585759-8qgkj" Jul 20 13:14:31.426093 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.425546 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f74585759-8qgkj" event={"ID":"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f","Type":"ContainerDied","Data":"c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7"} Jul 20 13:14:31.426093 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.425600 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f74585759-8qgkj" event={"ID":"6e5f5199-36a9-4c52-a91e-7a492a6f3e2f","Type":"ContainerDied","Data":"d7327b525855eaacf9bf520b8d497972f8123afe1375aa757b7d6478f3ddc36f"} Jul 20 13:14:31.426093 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.425624 2564 scope.go:117] "RemoveContainer" containerID="c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7" Jul 20 13:14:31.435196 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.435173 2564 scope.go:117] "RemoveContainer" containerID="c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7" Jul 20 13:14:31.435469 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:14:31.435447 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7\": container with ID starting with c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7 not found: ID does not exist" containerID="c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7" Jul 20 13:14:31.435517 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.435480 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7"} err="failed to get container status \"c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7\": rpc error: code = NotFound desc = could not find container \"c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7\": container with ID starting with c5ad74a19111279dad616d1e747fd23887dda55f41d1c2ededf701e648c44aa7 not found: ID does not exist" Jul 20 13:14:31.448031 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.447995 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f74585759-8qgkj"] Jul 20 13:14:31.449606 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.449587 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7f74585759-8qgkj"] Jul 20 13:14:31.936144 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:14:31.936112 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" path="/var/lib/kubelet/pods/6e5f5199-36a9-4c52-a91e-7a492a6f3e2f/volumes" Jul 20 13:16:29.865925 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:16:29.865873 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:16:29.866385 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:16:29.865945 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:16:29.869614 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:16:29.869596 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:16:29.869757 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:16:29.869677 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:21:29.894553 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:21:29.894519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:21:29.895156 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:21:29.894844 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:21:29.898971 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:21:29.898949 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:21:29.899114 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:21:29.899099 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:26:29.926215 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:26:29.926187 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:26:29.926750 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:26:29.926373 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:26:29.929920 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:26:29.929903 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:26:29.930347 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:26:29.930324 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:31:29.953904 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:31:29.953859 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:31:29.956147 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:31:29.956127 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:31:29.957848 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:31:29.957828 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:31:29.960342 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:31:29.960320 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:36:29.984185 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:36:29.984152 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:36:29.986915 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:36:29.986895 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:36:29.988428 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:36:29.988401 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:36:29.991236 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:36:29.991216 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:39:31.523956 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.523921 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb"] Jul 20 13:39:31.524473 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.524453 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" containerName="authorino" Jul 20 13:39:31.524521 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.524478 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" containerName="authorino" Jul 20 13:39:31.524576 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.524564 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5f5199-36a9-4c52-a91e-7a492a6f3e2f" containerName="authorino" Jul 20 13:39:31.527738 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.527720 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.531545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.531522 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fc9057\"/\"default-dockercfg-7dthm\"" Jul 20 13:39:31.531545 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.531535 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fc9057\"/\"kube-root-ca.crt\"" Jul 20 13:39:31.531726 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.531522 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fc9057\"/\"test-model-fc9057-kserve-self-signed-certs\"" Jul 20 13:39:31.531726 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.531572 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fc9057\"/\"openshift-service-ca.crt\"" Jul 20 13:39:31.536944 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.536920 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb"] Jul 20 13:39:31.568137 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.568107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-tmp-dir\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.568312 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.568145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2dd390-5e26-4dd4-9555-09ac438250d4-tls-certs\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.568312 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.568201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmtnd\" (UniqueName: \"kubernetes.io/projected/3f2dd390-5e26-4dd4-9555-09ac438250d4-kube-api-access-rmtnd\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.568312 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.568255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-model-cache\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.568312 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.568288 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-dshm\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.568495 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.568348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-home\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.669578 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.669539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-tmp-dir\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.669764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.669586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2dd390-5e26-4dd4-9555-09ac438250d4-tls-certs\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.669764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.669639 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtnd\" (UniqueName: \"kubernetes.io/projected/3f2dd390-5e26-4dd4-9555-09ac438250d4-kube-api-access-rmtnd\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.669764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.669684 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-model-cache\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.669764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.669711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-dshm\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.669764 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.669745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-home\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.670194 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.670159 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-tmp-dir\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.670322 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.670208 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-model-cache\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.670322 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.670225 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-home\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.671983 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.671962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-dshm\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.672147 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.672130 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2dd390-5e26-4dd4-9555-09ac438250d4-tls-certs\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.677614 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.677593 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmtnd\" (UniqueName: \"kubernetes.io/projected/3f2dd390-5e26-4dd4-9555-09ac438250d4-kube-api-access-rmtnd\") pod \"test-model-fc9057-kserve-c8f9468bf-xqdhb\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.838216 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.838123 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:31.973845 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:31.973811 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb"] Jul 20 13:39:31.977077 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:39:31.977050 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f2dd390_5e26_4dd4_9555_09ac438250d4.slice/crio-947b3584265418a92b641eabec102a009a322b8b6e29b0c2c83b9b656f387c66 WatchSource:0}: Error finding container 947b3584265418a92b641eabec102a009a322b8b6e29b0c2c83b9b656f387c66: Status 404 returned error can't find the container with id 947b3584265418a92b641eabec102a009a322b8b6e29b0c2c83b9b656f387c66 Jul 20 13:39:32.429081 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:32.429043 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" event={"ID":"3f2dd390-5e26-4dd4-9555-09ac438250d4","Type":"ContainerStarted","Data":"57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d"} Jul 20 13:39:32.429081 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:32.429092 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" event={"ID":"3f2dd390-5e26-4dd4-9555-09ac438250d4","Type":"ContainerStarted","Data":"947b3584265418a92b641eabec102a009a322b8b6e29b0c2c83b9b656f387c66"} Jul 20 13:39:32.447424 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:32.447369 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" podStartSLOduration=1.447354545 podStartE2EDuration="1.447354545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:39:32.445325814 +0000 UTC m=+2282.997218513" watchObservedRunningTime="2026-07-20 13:39:32.447354545 +0000 UTC m=+2282.999247244" Jul 20 13:39:41.839205 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:41.839160 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:41.839628 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:41.839221 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:41.851778 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:41.851752 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:42.489554 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:42.489515 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:39:56.894025 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:39:56.893988 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb"] Jul 20 13:40:11.914515 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:11.914453 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" podUID="3f2dd390-5e26-4dd4-9555-09ac438250d4" containerName="main" containerID="cri-o://57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d" gracePeriod=45 Jul 20 13:40:12.164951 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.164871 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:40:12.329936 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.329870 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-tmp-dir\") pod \"3f2dd390-5e26-4dd4-9555-09ac438250d4\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " Jul 20 13:40:12.329936 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.329943 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2dd390-5e26-4dd4-9555-09ac438250d4-tls-certs\") pod \"3f2dd390-5e26-4dd4-9555-09ac438250d4\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " Jul 20 13:40:12.330196 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.329991 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-home\") pod \"3f2dd390-5e26-4dd4-9555-09ac438250d4\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " Jul 20 13:40:12.330196 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330037 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-dshm\") pod \"3f2dd390-5e26-4dd4-9555-09ac438250d4\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " Jul 20 13:40:12.330196 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330071 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-model-cache\") pod \"3f2dd390-5e26-4dd4-9555-09ac438250d4\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " Jul 20 13:40:12.330196 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330091 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmtnd\" (UniqueName: \"kubernetes.io/projected/3f2dd390-5e26-4dd4-9555-09ac438250d4-kube-api-access-rmtnd\") pod \"3f2dd390-5e26-4dd4-9555-09ac438250d4\" (UID: \"3f2dd390-5e26-4dd4-9555-09ac438250d4\") " Jul 20 13:40:12.330406 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330213 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f2dd390-5e26-4dd4-9555-09ac438250d4" (UID: "3f2dd390-5e26-4dd4-9555-09ac438250d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:40:12.330406 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330291 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-home" (OuterVolumeSpecName: "home") pod "3f2dd390-5e26-4dd4-9555-09ac438250d4" (UID: "3f2dd390-5e26-4dd4-9555-09ac438250d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:40:12.330406 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330304 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f2dd390-5e26-4dd4-9555-09ac438250d4" (UID: "3f2dd390-5e26-4dd4-9555-09ac438250d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:40:12.330406 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.330332 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-tmp-dir\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:40:12.332310 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.332278 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-dshm" (OuterVolumeSpecName: "dshm") pod "3f2dd390-5e26-4dd4-9555-09ac438250d4" (UID: "3f2dd390-5e26-4dd4-9555-09ac438250d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:40:12.332310 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.332281 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2dd390-5e26-4dd4-9555-09ac438250d4-kube-api-access-rmtnd" (OuterVolumeSpecName: "kube-api-access-rmtnd") pod "3f2dd390-5e26-4dd4-9555-09ac438250d4" (UID: "3f2dd390-5e26-4dd4-9555-09ac438250d4"). InnerVolumeSpecName "kube-api-access-rmtnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:40:12.332449 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.332323 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f2dd390-5e26-4dd4-9555-09ac438250d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f2dd390-5e26-4dd4-9555-09ac438250d4" (UID: "3f2dd390-5e26-4dd4-9555-09ac438250d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:40:12.431645 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.431553 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-model-cache\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:40:12.431645 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.431592 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmtnd\" (UniqueName: \"kubernetes.io/projected/3f2dd390-5e26-4dd4-9555-09ac438250d4-kube-api-access-rmtnd\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:40:12.431645 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.431603 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f2dd390-5e26-4dd4-9555-09ac438250d4-tls-certs\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:40:12.431645 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.431612 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-home\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:40:12.431645 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.431619 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f2dd390-5e26-4dd4-9555-09ac438250d4-dshm\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:40:12.620466 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.620427 2564 generic.go:356] "Generic (PLEG): container finished" podID="3f2dd390-5e26-4dd4-9555-09ac438250d4" containerID="57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d" exitCode=0 Jul 20 13:40:12.620668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.620470 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" Jul 20 13:40:12.620668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.620512 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" event={"ID":"3f2dd390-5e26-4dd4-9555-09ac438250d4","Type":"ContainerDied","Data":"57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d"} Jul 20 13:40:12.620668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.620553 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb" event={"ID":"3f2dd390-5e26-4dd4-9555-09ac438250d4","Type":"ContainerDied","Data":"947b3584265418a92b641eabec102a009a322b8b6e29b0c2c83b9b656f387c66"} Jul 20 13:40:12.620668 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.620574 2564 scope.go:117] "RemoveContainer" containerID="57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d" Jul 20 13:40:12.629967 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.629952 2564 scope.go:117] "RemoveContainer" containerID="57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d" Jul 20 13:40:12.630213 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:40:12.630192 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d\": container with ID starting with 57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d not found: ID does not exist" containerID="57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d" Jul 20 13:40:12.630255 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.630223 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d"} err="failed to get container status \"57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d\": rpc error: code = NotFound desc = could not find container \"57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d\": container with ID starting with 57a0816f61c3ba5848edabbcf5ef3ac769b05d8c1f0b406ef0e7bca2dcc8fe5d not found: ID does not exist" Jul 20 13:40:12.644142 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.644114 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb"] Jul 20 13:40:12.645717 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:12.645697 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-fc9057/test-model-fc9057-kserve-c8f9468bf-xqdhb"] Jul 20 13:40:13.936165 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:40:13.936133 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f2dd390-5e26-4dd4-9555-09ac438250d4" path="/var/lib/kubelet/pods/3f2dd390-5e26-4dd4-9555-09ac438250d4/volumes" Jul 20 13:41:30.014749 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:41:30.014645 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:41:30.018847 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:41:30.018758 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:41:30.018847 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:41:30.018825 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:41:30.022908 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:41:30.022868 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:43:02.730271 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.730234 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd"] Jul 20 13:43:02.730741 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.730546 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2dd390-5e26-4dd4-9555-09ac438250d4" containerName="main" Jul 20 13:43:02.730741 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.730557 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2dd390-5e26-4dd4-9555-09ac438250d4" containerName="main" Jul 20 13:43:02.730741 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.730621 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2dd390-5e26-4dd4-9555-09ac438250d4" containerName="main" Jul 20 13:43:02.733675 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.733659 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.736305 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.736283 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"openshift-service-ca.crt\"" Jul 20 13:43:02.737640 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.737620 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"default-dockercfg-gx8z2\"" Jul 20 13:43:02.737729 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.737643 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"auth-test-model-a69896-kserve-self-signed-certs\"" Jul 20 13:43:02.737729 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.737645 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"kube-root-ca.crt\"" Jul 20 13:43:02.741589 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.741563 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd"] Jul 20 13:43:02.780244 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.780215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tmp-dir\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.780244 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.780247 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tls-certs\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.780442 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.780267 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-dshm\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.780442 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.780325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqj5g\" (UniqueName: \"kubernetes.io/projected/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-kube-api-access-pqj5g\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.780442 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.780381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-home\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.780548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.780448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-model-cache\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.880795 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.880760 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-home\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881008 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.880805 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-model-cache\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881008 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.880857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tmp-dir\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881008 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.880898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tls-certs\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881008 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.880929 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-dshm\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881008 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.880956 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqj5g\" (UniqueName: \"kubernetes.io/projected/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-kube-api-access-pqj5g\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881278 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.881229 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-home\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881323 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.881283 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-model-cache\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.881365 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.881311 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tmp-dir\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.883131 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.883112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-dshm\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.883503 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.883486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tls-certs\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:02.889902 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:02.889856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqj5g\" (UniqueName: \"kubernetes.io/projected/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-kube-api-access-pqj5g\") pod \"auth-test-model-a69896-kserve-6575dccdbc-fs7cd\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:03.043151 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:03.043070 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:03.177949 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:03.177923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd"] Jul 20 13:43:03.180046 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:43:03.180019 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccfb2832_41ba_4ef6_aebb_0fd79a21222d.slice/crio-8f262caa6e88ff643c19bfb27b37343a3832f9bcd9d11b264c623691d31e84ec WatchSource:0}: Error finding container 8f262caa6e88ff643c19bfb27b37343a3832f9bcd9d11b264c623691d31e84ec: Status 404 returned error can't find the container with id 8f262caa6e88ff643c19bfb27b37343a3832f9bcd9d11b264c623691d31e84ec Jul 20 13:43:03.421776 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:03.421689 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" event={"ID":"ccfb2832-41ba-4ef6-aebb-0fd79a21222d","Type":"ContainerStarted","Data":"4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422"} Jul 20 13:43:03.421776 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:03.421734 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" event={"ID":"ccfb2832-41ba-4ef6-aebb-0fd79a21222d","Type":"ContainerStarted","Data":"8f262caa6e88ff643c19bfb27b37343a3832f9bcd9d11b264c623691d31e84ec"} Jul 20 13:43:03.443776 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:03.443722 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" podStartSLOduration=1.443708253 podStartE2EDuration="1.443708253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:43:03.442144942 +0000 UTC m=+2493.994037652" watchObservedRunningTime="2026-07-20 13:43:03.443708253 +0000 UTC m=+2493.995600952" Jul 20 13:43:13.043970 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:13.043932 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:13.043970 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:13.043981 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:13.056445 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:13.056420 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:43:13.478887 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:43:13.478842 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:45:09.974682 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:09.974601 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd"] Jul 20 13:45:24.994161 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:24.994117 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" podUID="ccfb2832-41ba-4ef6-aebb-0fd79a21222d" containerName="main" containerID="cri-o://4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422" gracePeriod=45 Jul 20 13:45:25.243423 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.243394 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:45:25.419374 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419293 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-home\") pod \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " Jul 20 13:45:25.419374 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419368 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-model-cache\") pod \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " Jul 20 13:45:25.419609 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419395 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tmp-dir\") pod \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " Jul 20 13:45:25.419609 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419424 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tls-certs\") pod \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " Jul 20 13:45:25.419609 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419465 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-dshm\") pod \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " Jul 20 13:45:25.419609 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419544 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqj5g\" (UniqueName: \"kubernetes.io/projected/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-kube-api-access-pqj5g\") pod \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\" (UID: \"ccfb2832-41ba-4ef6-aebb-0fd79a21222d\") " Jul 20 13:45:25.419812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419646 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-home" (OuterVolumeSpecName: "home") pod "ccfb2832-41ba-4ef6-aebb-0fd79a21222d" (UID: "ccfb2832-41ba-4ef6-aebb-0fd79a21222d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:45:25.419812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419659 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-model-cache" (OuterVolumeSpecName: "model-cache") pod "ccfb2832-41ba-4ef6-aebb-0fd79a21222d" (UID: "ccfb2832-41ba-4ef6-aebb-0fd79a21222d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:45:25.419812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419681 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ccfb2832-41ba-4ef6-aebb-0fd79a21222d" (UID: "ccfb2832-41ba-4ef6-aebb-0fd79a21222d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:45:25.419970 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419833 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-model-cache\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:45:25.419970 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419852 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tmp-dir\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:45:25.419970 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.419865 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-home\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:45:25.421739 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.421714 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-dshm" (OuterVolumeSpecName: "dshm") pod "ccfb2832-41ba-4ef6-aebb-0fd79a21222d" (UID: "ccfb2832-41ba-4ef6-aebb-0fd79a21222d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:45:25.421855 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.421835 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ccfb2832-41ba-4ef6-aebb-0fd79a21222d" (UID: "ccfb2832-41ba-4ef6-aebb-0fd79a21222d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:45:25.421924 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.421840 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-kube-api-access-pqj5g" (OuterVolumeSpecName: "kube-api-access-pqj5g") pod "ccfb2832-41ba-4ef6-aebb-0fd79a21222d" (UID: "ccfb2832-41ba-4ef6-aebb-0fd79a21222d"). InnerVolumeSpecName "kube-api-access-pqj5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:45:25.520438 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.520393 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-tls-certs\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:45:25.520438 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.520431 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-dshm\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:45:25.520438 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:25.520445 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqj5g\" (UniqueName: \"kubernetes.io/projected/ccfb2832-41ba-4ef6-aebb-0fd79a21222d-kube-api-access-pqj5g\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:45:26.086184 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.086151 2564 generic.go:356] "Generic (PLEG): container finished" podID="ccfb2832-41ba-4ef6-aebb-0fd79a21222d" containerID="4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422" exitCode=0 Jul 20 13:45:26.086720 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.086191 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" Jul 20 13:45:26.086720 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.086194 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" event={"ID":"ccfb2832-41ba-4ef6-aebb-0fd79a21222d","Type":"ContainerDied","Data":"4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422"} Jul 20 13:45:26.086720 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.086235 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd" event={"ID":"ccfb2832-41ba-4ef6-aebb-0fd79a21222d","Type":"ContainerDied","Data":"8f262caa6e88ff643c19bfb27b37343a3832f9bcd9d11b264c623691d31e84ec"} Jul 20 13:45:26.086720 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.086256 2564 scope.go:117] "RemoveContainer" containerID="4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422" Jul 20 13:45:26.094941 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.094922 2564 scope.go:117] "RemoveContainer" containerID="4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422" Jul 20 13:45:26.095226 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:45:26.095207 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422\": container with ID starting with 4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422 not found: ID does not exist" containerID="4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422" Jul 20 13:45:26.095275 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.095235 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422"} err="failed to get container status \"4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422\": rpc error: code = NotFound desc = could not find container \"4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422\": container with ID starting with 4122e765460f0ada24d49039f69fbbaa4ec31f7faa7e62bb5aae533088311422 not found: ID does not exist" Jul 20 13:45:26.103658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.103636 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd"] Jul 20 13:45:26.107164 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:26.107138 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/auth-test-model-a69896-kserve-6575dccdbc-fs7cd"] Jul 20 13:45:27.936131 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:27.936100 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccfb2832-41ba-4ef6-aebb-0fd79a21222d" path="/var/lib/kubelet/pods/ccfb2832-41ba-4ef6-aebb-0fd79a21222d/volumes" Jul 20 13:45:32.942537 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.942494 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd"] Jul 20 13:45:32.943033 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.942979 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccfb2832-41ba-4ef6-aebb-0fd79a21222d" containerName="main" Jul 20 13:45:32.943033 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.942999 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccfb2832-41ba-4ef6-aebb-0fd79a21222d" containerName="main" Jul 20 13:45:32.943217 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.943110 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccfb2832-41ba-4ef6-aebb-0fd79a21222d" containerName="main" Jul 20 13:45:32.947546 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.947523 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:32.952325 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.952306 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"openshift-service-ca.crt\"" Jul 20 13:45:32.952658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.952636 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"sub-test-model-a69896-kserve-self-signed-certs\"" Jul 20 13:45:32.954143 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.953952 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"kube-root-ca.crt\"" Jul 20 13:45:32.954143 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.953978 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a69896\"/\"default-dockercfg-gx8z2\"" Jul 20 13:45:32.960738 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:32.960714 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd"] Jul 20 13:45:33.086841 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.086803 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce54c303-9ab3-4d79-b501-a2353fed4932-tls-certs\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.087039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.086848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-model-cache\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.087039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.086935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-home\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.087039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.087002 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-dshm\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.087039 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.087018 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-tmp-dir\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.087195 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.087095 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxdvx\" (UniqueName: \"kubernetes.io/projected/ce54c303-9ab3-4d79-b501-a2353fed4932-kube-api-access-lxdvx\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.188482 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.188446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce54c303-9ab3-4d79-b501-a2353fed4932-tls-certs\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.188658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.188497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-model-cache\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.188658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.188545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-home\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.188658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.188580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-dshm\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.188658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.188602 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-tmp-dir\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.188658 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.188635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxdvx\" (UniqueName: \"kubernetes.io/projected/ce54c303-9ab3-4d79-b501-a2353fed4932-kube-api-access-lxdvx\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.189058 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.189018 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-model-cache\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.189170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.189080 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-home\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.189170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.189102 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-tmp-dir\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.190894 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.190851 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-dshm\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.191097 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.191072 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce54c303-9ab3-4d79-b501-a2353fed4932-tls-certs\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.200614 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.200558 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxdvx\" (UniqueName: \"kubernetes.io/projected/ce54c303-9ab3-4d79-b501-a2353fed4932-kube-api-access-lxdvx\") pod \"sub-test-model-a69896-kserve-65885567cc-8nngd\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.256548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.256506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:33.392185 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:33.392123 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd"] Jul 20 13:45:33.394546 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:45:33.394517 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce54c303_9ab3_4d79_b501_a2353fed4932.slice/crio-74694285fbea55d28fab64ed18e351d97cdfcee106414bc2201939c4577c9953 WatchSource:0}: Error finding container 74694285fbea55d28fab64ed18e351d97cdfcee106414bc2201939c4577c9953: Status 404 returned error can't find the container with id 74694285fbea55d28fab64ed18e351d97cdfcee106414bc2201939c4577c9953 Jul 20 13:45:34.124436 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:34.124396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" event={"ID":"ce54c303-9ab3-4d79-b501-a2353fed4932","Type":"ContainerStarted","Data":"d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137"} Jul 20 13:45:34.124436 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:34.124442 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" event={"ID":"ce54c303-9ab3-4d79-b501-a2353fed4932","Type":"ContainerStarted","Data":"74694285fbea55d28fab64ed18e351d97cdfcee106414bc2201939c4577c9953"} Jul 20 13:45:34.144972 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:34.144917 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" podStartSLOduration=2.144901337 podStartE2EDuration="2.144901337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:45:34.142706545 +0000 UTC m=+2644.694599246" watchObservedRunningTime="2026-07-20 13:45:34.144901337 +0000 UTC m=+2644.696794033" Jul 20 13:45:43.256907 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:43.256836 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:43.257475 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:43.256929 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:43.269815 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:43.269781 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:45:44.187061 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:45:44.187030 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:46:27.643801 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:27.643718 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd"] Jul 20 13:46:30.043200 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:30.043092 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:46:30.048158 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:30.047275 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:46:30.049408 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:30.049387 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:46:30.053408 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:30.053386 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rjm7q_88320fe0-e206-431b-ac3e-8ab93c4ab9ac/ovn-acl-logging/0.log" Jul 20 13:46:36.434230 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.434198 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc"] Jul 20 13:46:36.438191 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.438170 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.441170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.441143 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5e9fd6\"/\"openshift-service-ca.crt\"" Jul 20 13:46:36.442341 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.442317 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5e9fd6\"/\"rate-test-model-5e9fd6-kserve-self-signed-certs\"" Jul 20 13:46:36.442435 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.442396 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5e9fd6\"/\"default-dockercfg-gzftg\"" Jul 20 13:46:36.442501 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.442457 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5e9fd6\"/\"kube-root-ca.crt\"" Jul 20 13:46:36.448074 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.448053 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc"] Jul 20 13:46:36.533748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.533711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-tmp-dir\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.533748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.533753 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxlj\" (UniqueName: \"kubernetes.io/projected/ba78c6be-9787-4786-bbbd-008f929fb4c5-kube-api-access-gtxlj\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.534020 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.533829 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-home\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.534020 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.533865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-model-cache\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.534020 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.533926 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba78c6be-9787-4786-bbbd-008f929fb4c5-tls-certs\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.534020 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.533970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-dshm\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.634522 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.634485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-model-cache\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.634522 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.634521 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba78c6be-9787-4786-bbbd-008f929fb4c5-tls-certs\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.634768 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.634548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-dshm\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.634768 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.634596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-tmp-dir\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.634768 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.634615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxlj\" (UniqueName: \"kubernetes.io/projected/ba78c6be-9787-4786-bbbd-008f929fb4c5-kube-api-access-gtxlj\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.634768 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.634671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-home\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.635116 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.635089 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-home\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.635182 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.635098 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-model-cache\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.635182 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.635144 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-tmp-dir\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.636950 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.636918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-dshm\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.637099 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.637085 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba78c6be-9787-4786-bbbd-008f929fb4c5-tls-certs\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.648729 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.648705 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxlj\" (UniqueName: \"kubernetes.io/projected/ba78c6be-9787-4786-bbbd-008f929fb4c5-kube-api-access-gtxlj\") pod \"rate-test-model-5e9fd6-kserve-67cb648975-875tc\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.748037 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.748009 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:36.886625 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:36.886602 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc"] Jul 20 13:46:36.888433 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:46:36.888400 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba78c6be_9787_4786_bbbd_008f929fb4c5.slice/crio-c5fdd4d1942f6874c68e13d8b9725d460feac5e6bb28f850e7217a747ecb7a9b WatchSource:0}: Error finding container c5fdd4d1942f6874c68e13d8b9725d460feac5e6bb28f850e7217a747ecb7a9b: Status 404 returned error can't find the container with id c5fdd4d1942f6874c68e13d8b9725d460feac5e6bb28f850e7217a747ecb7a9b Jul 20 13:46:37.440927 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:37.440869 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" event={"ID":"ba78c6be-9787-4786-bbbd-008f929fb4c5","Type":"ContainerStarted","Data":"6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679"} Jul 20 13:46:37.440927 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:37.440930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" event={"ID":"ba78c6be-9787-4786-bbbd-008f929fb4c5","Type":"ContainerStarted","Data":"c5fdd4d1942f6874c68e13d8b9725d460feac5e6bb28f850e7217a747ecb7a9b"} Jul 20 13:46:37.480685 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:37.480631 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" podStartSLOduration=1.480614976 podStartE2EDuration="1.480614976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 13:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:46:37.478665145 +0000 UTC m=+2708.030557855" watchObservedRunningTime="2026-07-20 13:46:37.480614976 +0000 UTC m=+2708.032507675" Jul 20 13:46:42.664152 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.664113 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" podUID="ce54c303-9ab3-4d79-b501-a2353fed4932" containerName="main" containerID="cri-o://d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137" gracePeriod=45 Jul 20 13:46:42.918712 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.918651 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:46:42.983153 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983118 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-model-cache\") pod \"ce54c303-9ab3-4d79-b501-a2353fed4932\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " Jul 20 13:46:42.983326 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983181 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-dshm\") pod \"ce54c303-9ab3-4d79-b501-a2353fed4932\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " Jul 20 13:46:42.983326 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983200 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-home\") pod \"ce54c303-9ab3-4d79-b501-a2353fed4932\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " Jul 20 13:46:42.983326 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983237 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxdvx\" (UniqueName: \"kubernetes.io/projected/ce54c303-9ab3-4d79-b501-a2353fed4932-kube-api-access-lxdvx\") pod \"ce54c303-9ab3-4d79-b501-a2353fed4932\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " Jul 20 13:46:42.983326 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983261 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce54c303-9ab3-4d79-b501-a2353fed4932-tls-certs\") pod \"ce54c303-9ab3-4d79-b501-a2353fed4932\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " Jul 20 13:46:42.983326 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983288 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-tmp-dir\") pod \"ce54c303-9ab3-4d79-b501-a2353fed4932\" (UID: \"ce54c303-9ab3-4d79-b501-a2353fed4932\") " Jul 20 13:46:42.983600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983444 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce54c303-9ab3-4d79-b501-a2353fed4932" (UID: "ce54c303-9ab3-4d79-b501-a2353fed4932"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:46:42.983600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983520 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-home" (OuterVolumeSpecName: "home") pod "ce54c303-9ab3-4d79-b501-a2353fed4932" (UID: "ce54c303-9ab3-4d79-b501-a2353fed4932"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:46:42.983600 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.983584 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce54c303-9ab3-4d79-b501-a2353fed4932" (UID: "ce54c303-9ab3-4d79-b501-a2353fed4932"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:46:42.985387 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.985358 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce54c303-9ab3-4d79-b501-a2353fed4932-kube-api-access-lxdvx" (OuterVolumeSpecName: "kube-api-access-lxdvx") pod "ce54c303-9ab3-4d79-b501-a2353fed4932" (UID: "ce54c303-9ab3-4d79-b501-a2353fed4932"). InnerVolumeSpecName "kube-api-access-lxdvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:46:42.985387 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.985364 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce54c303-9ab3-4d79-b501-a2353fed4932-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce54c303-9ab3-4d79-b501-a2353fed4932" (UID: "ce54c303-9ab3-4d79-b501-a2353fed4932"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:46:42.985525 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:42.985418 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-dshm" (OuterVolumeSpecName: "dshm") pod "ce54c303-9ab3-4d79-b501-a2353fed4932" (UID: "ce54c303-9ab3-4d79-b501-a2353fed4932"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:46:43.084029 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.083991 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxdvx\" (UniqueName: \"kubernetes.io/projected/ce54c303-9ab3-4d79-b501-a2353fed4932-kube-api-access-lxdvx\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:46:43.084029 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.084023 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce54c303-9ab3-4d79-b501-a2353fed4932-tls-certs\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:46:43.084029 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.084038 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-tmp-dir\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:46:43.084265 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.084050 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-model-cache\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:46:43.084265 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.084061 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-dshm\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:46:43.084265 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.084070 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce54c303-9ab3-4d79-b501-a2353fed4932-home\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:46:43.470083 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.470051 2564 generic.go:356] "Generic (PLEG): container finished" podID="ce54c303-9ab3-4d79-b501-a2353fed4932" containerID="d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137" exitCode=0 Jul 20 13:46:43.470258 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.470092 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" Jul 20 13:46:43.470258 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.470097 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" event={"ID":"ce54c303-9ab3-4d79-b501-a2353fed4932","Type":"ContainerDied","Data":"d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137"} Jul 20 13:46:43.470258 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.470129 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd" event={"ID":"ce54c303-9ab3-4d79-b501-a2353fed4932","Type":"ContainerDied","Data":"74694285fbea55d28fab64ed18e351d97cdfcee106414bc2201939c4577c9953"} Jul 20 13:46:43.470258 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.470147 2564 scope.go:117] "RemoveContainer" containerID="d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137" Jul 20 13:46:43.478949 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.478931 2564 scope.go:117] "RemoveContainer" containerID="d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137" Jul 20 13:46:43.479187 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:46:43.479171 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137\": container with ID starting with d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137 not found: ID does not exist" containerID="d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137" Jul 20 13:46:43.479239 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.479195 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137"} err="failed to get container status \"d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137\": rpc error: code = NotFound desc = could not find container \"d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137\": container with ID starting with d3b280047928fcbe7a44cb3ee41a9a359e6fd49b3e2f2c67d3e17d0e0ed0d137 not found: ID does not exist" Jul 20 13:46:43.501975 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.501942 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd"] Jul 20 13:46:43.511721 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.511695 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a69896/sub-test-model-a69896-kserve-65885567cc-8nngd"] Jul 20 13:46:43.936426 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:43.936399 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce54c303-9ab3-4d79-b501-a2353fed4932" path="/var/lib/kubelet/pods/ce54c303-9ab3-4d79-b501-a2353fed4932/volumes" Jul 20 13:46:46.748854 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:46.748817 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:46.749296 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:46.748873 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:46.761640 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:46.761617 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:46:47.501548 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:46:47.501517 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:47:36.363015 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:36.362981 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc"] Jul 20 13:47:50.203828 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.203792 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7719-140176945222976"] Jul 20 13:47:50.204318 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.204157 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce54c303-9ab3-4d79-b501-a2353fed4932" containerName="main" Jul 20 13:47:50.204318 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.204170 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce54c303-9ab3-4d79-b501-a2353fed4932" containerName="main" Jul 20 13:47:50.204318 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.204227 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce54c303-9ab3-4d79-b501-a2353fed4932" containerName="main" Jul 20 13:47:50.207139 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.207124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:50.210066 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.210048 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hms82\"" Jul 20 13:47:50.215343 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.215322 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7719-140176945222976"] Jul 20 13:47:50.251256 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.251228 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrlpt\" (UniqueName: \"kubernetes.io/projected/630a91a7-769d-4d7f-b404-0b2b2b456f33-kube-api-access-vrlpt\") pod \"test-curl-7719-140176945222976\" (UID: \"630a91a7-769d-4d7f-b404-0b2b2b456f33\") " pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:50.352222 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.352186 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrlpt\" (UniqueName: \"kubernetes.io/projected/630a91a7-769d-4d7f-b404-0b2b2b456f33-kube-api-access-vrlpt\") pod \"test-curl-7719-140176945222976\" (UID: \"630a91a7-769d-4d7f-b404-0b2b2b456f33\") " pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:50.365015 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.364989 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrlpt\" (UniqueName: \"kubernetes.io/projected/630a91a7-769d-4d7f-b404-0b2b2b456f33-kube-api-access-vrlpt\") pod \"test-curl-7719-140176945222976\" (UID: \"630a91a7-769d-4d7f-b404-0b2b2b456f33\") " pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:50.516637 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.516557 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:50.659700 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.659672 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7719-140176945222976"] Jul 20 13:47:50.661511 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:47:50.661485 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod630a91a7_769d_4d7f_b404_0b2b2b456f33.slice/crio-37e40c8484c31236f8cf8d2111a1954987823f48a94d6e6c3d22ee9b0a724d23 WatchSource:0}: Error finding container 37e40c8484c31236f8cf8d2111a1954987823f48a94d6e6c3d22ee9b0a724d23: Status 404 returned error can't find the container with id 37e40c8484c31236f8cf8d2111a1954987823f48a94d6e6c3d22ee9b0a724d23 Jul 20 13:47:50.662672 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.662653 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 13:47:50.799606 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:50.799533 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7719-140176945222976" event={"ID":"630a91a7-769d-4d7f-b404-0b2b2b456f33","Type":"ContainerStarted","Data":"37e40c8484c31236f8cf8d2111a1954987823f48a94d6e6c3d22ee9b0a724d23"} Jul 20 13:47:51.382860 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.382828 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" podUID="ba78c6be-9787-4786-bbbd-008f929fb4c5" containerName="main" containerID="cri-o://6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679" gracePeriod=45 Jul 20 13:47:51.630402 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.630373 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:47:51.662939 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.662917 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-model-cache\") pod \"ba78c6be-9787-4786-bbbd-008f929fb4c5\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " Jul 20 13:47:51.663070 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.662974 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtxlj\" (UniqueName: \"kubernetes.io/projected/ba78c6be-9787-4786-bbbd-008f929fb4c5-kube-api-access-gtxlj\") pod \"ba78c6be-9787-4786-bbbd-008f929fb4c5\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " Jul 20 13:47:51.663070 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.662994 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-dshm\") pod \"ba78c6be-9787-4786-bbbd-008f929fb4c5\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " Jul 20 13:47:51.663070 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663049 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-tmp-dir\") pod \"ba78c6be-9787-4786-bbbd-008f929fb4c5\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " Jul 20 13:47:51.663214 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663075 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-home\") pod \"ba78c6be-9787-4786-bbbd-008f929fb4c5\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " Jul 20 13:47:51.663214 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663131 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba78c6be-9787-4786-bbbd-008f929fb4c5-tls-certs\") pod \"ba78c6be-9787-4786-bbbd-008f929fb4c5\" (UID: \"ba78c6be-9787-4786-bbbd-008f929fb4c5\") " Jul 20 13:47:51.663214 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663187 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba78c6be-9787-4786-bbbd-008f929fb4c5" (UID: "ba78c6be-9787-4786-bbbd-008f929fb4c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:47:51.663369 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663286 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba78c6be-9787-4786-bbbd-008f929fb4c5" (UID: "ba78c6be-9787-4786-bbbd-008f929fb4c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:47:51.663369 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663348 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-home" (OuterVolumeSpecName: "home") pod "ba78c6be-9787-4786-bbbd-008f929fb4c5" (UID: "ba78c6be-9787-4786-bbbd-008f929fb4c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:47:51.663483 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663458 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-tmp-dir\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:51.663539 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663482 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-home\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:51.663539 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.663497 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-model-cache\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:51.665176 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.665148 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-dshm" (OuterVolumeSpecName: "dshm") pod "ba78c6be-9787-4786-bbbd-008f929fb4c5" (UID: "ba78c6be-9787-4786-bbbd-008f929fb4c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 13:47:51.665276 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.665220 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba78c6be-9787-4786-bbbd-008f929fb4c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba78c6be-9787-4786-bbbd-008f929fb4c5" (UID: "ba78c6be-9787-4786-bbbd-008f929fb4c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 13:47:51.665507 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.665491 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba78c6be-9787-4786-bbbd-008f929fb4c5-kube-api-access-gtxlj" (OuterVolumeSpecName: "kube-api-access-gtxlj") pod "ba78c6be-9787-4786-bbbd-008f929fb4c5" (UID: "ba78c6be-9787-4786-bbbd-008f929fb4c5"). InnerVolumeSpecName "kube-api-access-gtxlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:47:51.764701 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.764668 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba78c6be-9787-4786-bbbd-008f929fb4c5-tls-certs\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:51.764701 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.764696 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtxlj\" (UniqueName: \"kubernetes.io/projected/ba78c6be-9787-4786-bbbd-008f929fb4c5-kube-api-access-gtxlj\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:51.764701 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.764705 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba78c6be-9787-4786-bbbd-008f929fb4c5-dshm\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:51.805449 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.805416 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba78c6be-9787-4786-bbbd-008f929fb4c5" containerID="6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679" exitCode=0 Jul 20 13:47:51.805587 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.805467 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" Jul 20 13:47:51.805587 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.805510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" event={"ID":"ba78c6be-9787-4786-bbbd-008f929fb4c5","Type":"ContainerDied","Data":"6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679"} Jul 20 13:47:51.805587 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.805558 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc" event={"ID":"ba78c6be-9787-4786-bbbd-008f929fb4c5","Type":"ContainerDied","Data":"c5fdd4d1942f6874c68e13d8b9725d460feac5e6bb28f850e7217a747ecb7a9b"} Jul 20 13:47:51.805587 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.805573 2564 scope.go:117] "RemoveContainer" containerID="6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679" Jul 20 13:47:51.807186 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.807164 2564 generic.go:356] "Generic (PLEG): container finished" podID="630a91a7-769d-4d7f-b404-0b2b2b456f33" containerID="4089e960e9a61b00d3b676ec87e76b3d056bcc288aaa4ad5a611f48ab7b5b358" exitCode=0 Jul 20 13:47:51.807278 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.807206 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7719-140176945222976" event={"ID":"630a91a7-769d-4d7f-b404-0b2b2b456f33","Type":"ContainerDied","Data":"4089e960e9a61b00d3b676ec87e76b3d056bcc288aaa4ad5a611f48ab7b5b358"} Jul 20 13:47:51.814703 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.814682 2564 scope.go:117] "RemoveContainer" containerID="6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679" Jul 20 13:47:51.814983 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:47:51.814961 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679\": container with ID starting with 6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679 not found: ID does not exist" containerID="6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679" Jul 20 13:47:51.815063 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.814989 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679"} err="failed to get container status \"6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679\": rpc error: code = NotFound desc = could not find container \"6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679\": container with ID starting with 6f87641d95d004e388d7e769e42db81427355fb8b9d521b3e8ca0c5955105679 not found: ID does not exist" Jul 20 13:47:51.852787 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.852762 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc"] Jul 20 13:47:51.857048 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.857025 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5e9fd6/rate-test-model-5e9fd6-kserve-67cb648975-875tc"] Jul 20 13:47:51.935442 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:51.935422 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba78c6be-9787-4786-bbbd-008f929fb4c5" path="/var/lib/kubelet/pods/ba78c6be-9787-4786-bbbd-008f929fb4c5/volumes" Jul 20 13:47:52.941612 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:52.941583 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:52.974997 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:52.974967 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrlpt\" (UniqueName: \"kubernetes.io/projected/630a91a7-769d-4d7f-b404-0b2b2b456f33-kube-api-access-vrlpt\") pod \"630a91a7-769d-4d7f-b404-0b2b2b456f33\" (UID: \"630a91a7-769d-4d7f-b404-0b2b2b456f33\") " Jul 20 13:47:52.977118 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:52.977091 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/630a91a7-769d-4d7f-b404-0b2b2b456f33-kube-api-access-vrlpt" (OuterVolumeSpecName: "kube-api-access-vrlpt") pod "630a91a7-769d-4d7f-b404-0b2b2b456f33" (UID: "630a91a7-769d-4d7f-b404-0b2b2b456f33"). InnerVolumeSpecName "kube-api-access-vrlpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 13:47:53.001114 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.001093 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7719-140176945222976_630a91a7-769d-4d7f-b404-0b2b2b456f33/test-curl-7719-140176945222976/0.log" Jul 20 13:47:53.034202 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.034173 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7719-140176945222976"] Jul 20 13:47:53.044516 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.044486 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7719-140176945222976"] Jul 20 13:47:53.076249 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.076168 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrlpt\" (UniqueName: \"kubernetes.io/projected/630a91a7-769d-4d7f-b404-0b2b2b456f33-kube-api-access-vrlpt\") on node \"ip-10-0-131-198.ec2.internal\" DevicePath \"\"" Jul 20 13:47:53.819797 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.819769 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37e40c8484c31236f8cf8d2111a1954987823f48a94d6e6c3d22ee9b0a724d23" Jul 20 13:47:53.819797 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.819780 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:53.830483 ip-10-0-131-198 kubenswrapper[2564]: E0720 13:47:53.830454 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7719-140176945222976\" is forbidden: User \"system:node:ip-10-0-131-198.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-198.ec2.internal' and this object" podUID="630a91a7-769d-4d7f-b404-0b2b2b456f33" pod="opendatahub/test-curl-7719-140176945222976" Jul 20 13:47:53.938604 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:47:53.938572 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="630a91a7-769d-4d7f-b404-0b2b2b456f33" path="/var/lib/kubelet/pods/630a91a7-769d-4d7f-b404-0b2b2b456f33/volumes" Jul 20 13:48:49.250041 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:48:49.250010 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d755cbb74-4ghj5_4d81ca6d-32e6-4128-adfe-49c890a6028f/authorino/0.log" Jul 20 13:48:55.566628 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:48:55.566592 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jq7bz_eeb1feda-4d8e-4787-916f-df9659c3ca6d/server/0.log" Jul 20 13:48:55.695333 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:48:55.695303 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2xtgl_0590f04d-997c-474a-91ad-8b13d74f296f/manager/0.log" Jul 20 13:48:55.947182 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:48:55.947150 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85766d874c-dbdzn_61f1ea27-3445-4f8f-9f59-bfa7978b15c9/copy-manifests/0.log" Jul 20 13:48:55.954645 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:48:55.954616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85766d874c-dbdzn_61f1ea27-3445-4f8f-9f59-bfa7978b15c9/manager/0.log" Jul 20 13:48:59.814714 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:48:59.814686 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d755cbb74-4ghj5_4d81ca6d-32e6-4128-adfe-49c890a6028f/authorino/0.log" Jul 20 13:49:00.178242 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:00.178172 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dvvwh_d440c203-acea-433e-8f98-dbc53192af42/kuadrant-console-plugin/0.log" Jul 20 13:49:01.385254 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:01.385223 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bb65f5cb7-z9wmg_f4b86a98-206e-4b56-8bbc-c427af1bb7e1/kube-auth-proxy/0.log" Jul 20 13:49:01.945518 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:01.945484 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655955bbb-7fznj_2116eb31-99a1-4ab5-848e-ebf5e6c4f938/router/0.log" Jul 20 13:49:02.606380 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:02.606346 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-m74ps_51e9f150-09e9-4a04-8bc1-7b362155b57d/main/0.log" Jul 20 13:49:02.848148 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:02.848106 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xd822_d760ff6e-2e59-4966-af23-be88740bc841/main/0.log" Jul 20 13:49:02.970136 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:02.970113 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-f8dpr_08c98ed7-3359-4c23-9433-7abcd40cd16e/main/0.log" Jul 20 13:49:12.243758 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:12.243726 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xcmtb_3791d792-594b-43fa-a1af-3ec0bf0aac2d/global-pull-secret-syncer/0.log" Jul 20 13:49:12.362033 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:12.362002 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qhscq_15d906bc-6eff-4da3-a415-2621ca34552b/konnectivity-agent/0.log" Jul 20 13:49:12.385391 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:12.385368 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-198.ec2.internal_e160140d4aff5bc366703726e04ba72a/haproxy/0.log" Jul 20 13:49:18.081371 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:18.081337 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d755cbb74-4ghj5_4d81ca6d-32e6-4128-adfe-49c890a6028f/authorino/0.log" Jul 20 13:49:18.225441 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:18.225409 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dvvwh_d440c203-acea-433e-8f98-dbc53192af42/kuadrant-console-plugin/0.log" Jul 20 13:49:20.519212 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:20.519168 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-mwnjl_9f66766f-2ab9-433d-aee8-fda507d5c768/cluster-monitoring-operator/0.log" Jul 20 13:49:20.840534 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:20.840456 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wtlq8_f42e17cf-729e-45d4-b9bf-63ef3c62d36e/node-exporter/0.log" Jul 20 13:49:20.862998 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:20.862975 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wtlq8_f42e17cf-729e-45d4-b9bf-63ef3c62d36e/kube-rbac-proxy/0.log" Jul 20 13:49:20.888828 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:20.888794 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wtlq8_f42e17cf-729e-45d4-b9bf-63ef3c62d36e/init-textfile/0.log" Jul 20 13:49:21.958514 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958435 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj"] Jul 20 13:49:21.958873 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958751 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba78c6be-9787-4786-bbbd-008f929fb4c5" containerName="main" Jul 20 13:49:21.958873 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958762 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba78c6be-9787-4786-bbbd-008f929fb4c5" containerName="main" Jul 20 13:49:21.958873 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958774 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="630a91a7-769d-4d7f-b404-0b2b2b456f33" containerName="test-curl-7719-140176945222976" Jul 20 13:49:21.958873 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958780 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="630a91a7-769d-4d7f-b404-0b2b2b456f33" containerName="test-curl-7719-140176945222976" Jul 20 13:49:21.958873 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958835 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="630a91a7-769d-4d7f-b404-0b2b2b456f33" containerName="test-curl-7719-140176945222976" Jul 20 13:49:21.958873 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.958846 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba78c6be-9787-4786-bbbd-008f929fb4c5" containerName="main" Jul 20 13:49:21.961738 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.961723 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:21.964265 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.964241 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rqnpn\"/\"kube-root-ca.crt\"" Jul 20 13:49:21.965380 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.965364 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rqnpn\"/\"openshift-service-ca.crt\"" Jul 20 13:49:21.965454 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.965385 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rqnpn\"/\"default-dockercfg-9dmdv\"" Jul 20 13:49:21.969997 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:21.969972 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj"] Jul 20 13:49:22.140728 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.140689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-podres\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.140949 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.140757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-proc\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.140949 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.140807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfzbh\" (UniqueName: \"kubernetes.io/projected/d470fc3b-c624-4b21-a1db-c0026dcc76d9-kube-api-access-nfzbh\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.140949 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.140834 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-sys\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.141098 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.140985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-lib-modules\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-lib-modules\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-podres\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242170 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-proc\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfzbh\" (UniqueName: \"kubernetes.io/projected/d470fc3b-c624-4b21-a1db-c0026dcc76d9-kube-api-access-nfzbh\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-sys\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242274 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-podres\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242294 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-proc\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242278 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-lib-modules\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.242431 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.242321 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d470fc3b-c624-4b21-a1db-c0026dcc76d9-sys\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.251089 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.251069 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfzbh\" (UniqueName: \"kubernetes.io/projected/d470fc3b-c624-4b21-a1db-c0026dcc76d9-kube-api-access-nfzbh\") pod \"perf-node-gather-daemonset-bwhqj\" (UID: \"d470fc3b-c624-4b21-a1db-c0026dcc76d9\") " pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.272978 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.272953 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:22.401070 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:22.401042 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj"] Jul 20 13:49:22.402936 ip-10-0-131-198 kubenswrapper[2564]: W0720 13:49:22.402902 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd470fc3b_c624_4b21_a1db_c0026dcc76d9.slice/crio-12a03b2745e86991a3a7d965fbf5b87a66504a408282ba60203c01a127a7eda2 WatchSource:0}: Error finding container 12a03b2745e86991a3a7d965fbf5b87a66504a408282ba60203c01a127a7eda2: Status 404 returned error can't find the container with id 12a03b2745e86991a3a7d965fbf5b87a66504a408282ba60203c01a127a7eda2 Jul 20 13:49:23.109492 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:23.109459 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/1.log" Jul 20 13:49:23.113963 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:23.113943 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nhh9k_212913d4-c0a2-4d4c-a2fc-94c38b2be88f/console-operator/2.log" Jul 20 13:49:23.234318 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:23.234284 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" event={"ID":"d470fc3b-c624-4b21-a1db-c0026dcc76d9","Type":"ContainerStarted","Data":"7cf3c9c05b74ca0d6882deb282f3b4452b15830bc857791ce908835a31106110"} Jul 20 13:49:23.234496 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:23.234333 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" event={"ID":"d470fc3b-c624-4b21-a1db-c0026dcc76d9","Type":"ContainerStarted","Data":"12a03b2745e86991a3a7d965fbf5b87a66504a408282ba60203c01a127a7eda2"} Jul 20 13:49:23.234496 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:23.234363 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:23.252514 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:23.252467 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" podStartSLOduration=2.252451434 podStartE2EDuration="2.252451434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 13:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 13:49:23.251200112 +0000 UTC m=+2873.803092832" watchObservedRunningTime="2026-07-20 13:49:23.252451434 +0000 UTC m=+2873.804344132" Jul 20 13:49:24.212126 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:24.212099 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-s4j7s_ddaf4c2e-4bab-4ce0-8185-f95df74b5369/volume-data-source-validator/0.log" Jul 20 13:49:25.420113 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:25.420087 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wgngz_8f037b6c-1ab5-453f-a697-9366f4eaebb2/dns/0.log" Jul 20 13:49:25.459812 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:25.459787 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wgngz_8f037b6c-1ab5-453f-a697-9366f4eaebb2/kube-rbac-proxy/0.log" Jul 20 13:49:25.498130 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:25.498100 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6klgv_65a6fc4f-a16e-45bf-80c5-bb971c414db4/dns-node-resolver/0.log" Jul 20 13:49:26.146611 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:26.146581 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c6kfj_0a87dc87-258f-47ea-b5a4-1e5656f6a80c/node-ca/0.log" Jul 20 13:49:27.310501 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:27.310470 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bb65f5cb7-z9wmg_f4b86a98-206e-4b56-8bbc-c427af1bb7e1/kube-auth-proxy/0.log" Jul 20 13:49:27.455524 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:27.455496 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655955bbb-7fznj_2116eb31-99a1-4ab5-848e-ebf5e6c4f938/router/0.log" Jul 20 13:49:28.175481 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:28.175450 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-69txc_f4e9167d-2e5c-45e8-ac2d-c8f85981cd8c/serve-healthcheck-canary/0.log" Jul 20 13:49:28.682176 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:28.682142 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-jnvhs_50bbd98b-5610-4ad4-a413-3e4eec07581e/insights-operator/0.log" Jul 20 13:49:28.682837 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:28.682821 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-jnvhs_50bbd98b-5610-4ad4-a413-3e4eec07581e/insights-operator/1.log" Jul 20 13:49:28.854748 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:28.854721 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njlnn_a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9/kube-rbac-proxy/0.log" Jul 20 13:49:28.876346 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:28.876324 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njlnn_a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9/exporter/0.log" Jul 20 13:49:28.898389 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:28.898365 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njlnn_a2de0c37-7cb8-4b6c-b05b-ca279b6c6cf9/extractor/0.log" Jul 20 13:49:29.248042 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:29.248017 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rqnpn/perf-node-gather-daemonset-bwhqj" Jul 20 13:49:31.499686 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:31.499654 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jq7bz_eeb1feda-4d8e-4787-916f-df9659c3ca6d/server/0.log" Jul 20 13:49:31.568779 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:31.568707 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2xtgl_0590f04d-997c-474a-91ad-8b13d74f296f/manager/0.log" Jul 20 13:49:31.653403 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:31.653379 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85766d874c-dbdzn_61f1ea27-3445-4f8f-9f59-bfa7978b15c9/manager/0.log" Jul 20 13:49:31.678035 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:31.678010 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85766d874c-dbdzn_61f1ea27-3445-4f8f-9f59-bfa7978b15c9/copy-manifests/0.log" Jul 20 13:49:32.731971 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:32.731941 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bb65f5cb7-z9wmg_f4b86a98-206e-4b56-8bbc-c427af1bb7e1/kube-auth-proxy/0.log" Jul 20 13:49:32.839583 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:32.839554 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655955bbb-7fznj_2116eb31-99a1-4ab5-848e-ebf5e6c4f938/router/0.log" Jul 20 13:49:33.034768 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:33.034685 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84676b455f-njj7p_f7ccd0f0-eb96-4aaa-b0e3-ac1a8b7df66b/manager/0.log" Jul 20 13:49:34.197921 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:34.197860 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bb65f5cb7-z9wmg_f4b86a98-206e-4b56-8bbc-c427af1bb7e1/kube-auth-proxy/0.log" Jul 20 13:49:34.312070 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:34.312045 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5655955bbb-7fznj_2116eb31-99a1-4ab5-848e-ebf5e6c4f938/router/0.log" Jul 20 13:49:38.445527 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:38.445497 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-cfqrm_031c9773-aee4-42f0-ab1e-68925940613b/kube-storage-version-migrator-operator/1.log" Jul 20 13:49:38.446307 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:38.446286 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-cfqrm_031c9773-aee4-42f0-ab1e-68925940613b/kube-storage-version-migrator-operator/0.log" Jul 20 13:49:39.528537 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.528508 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7lvr4_080c0903-adad-4f79-ac46-d12d29c75d63/kube-multus/0.log" Jul 20 13:49:39.751190 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.751162 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/kube-multus-additional-cni-plugins/0.log" Jul 20 13:49:39.772488 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.772467 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/egress-router-binary-copy/0.log" Jul 20 13:49:39.792819 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.792749 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/cni-plugins/0.log" Jul 20 13:49:39.813564 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.813543 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/bond-cni-plugin/0.log" Jul 20 13:49:39.833270 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.833249 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/routeoverride-cni/0.log" Jul 20 13:49:39.854209 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.854189 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/whereabouts-cni-bincopy/0.log" Jul 20 13:49:39.877378 ip-10-0-131-198 kubenswrapper[2564]: I0720 13:49:39.877354 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qff7p_70fcd13b-8c85-4803-aabc-93c759f1f8d7/whereabouts-cni/0.log"