Jul 20 15:26:47.926065 ip-10-0-141-150 systemd[1]: Starting Kubernetes Kubelet... Jul 20 15:26:48.332147 ip-10-0-141-150 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 15:26:48.332147 ip-10-0-141-150 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 15:26:48.332147 ip-10-0-141-150 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 15:26:48.332147 ip-10-0-141-150 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 15:26:48.332147 ip-10-0-141-150 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 15:26:48.334969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.334883 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 15:26:48.339119 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339099 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:26:48.339119 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339119 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339123 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339127 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339130 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339133 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339136 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339140 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339143 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339145 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339148 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339153 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339157 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339160 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339168 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339171 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339175 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339177 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339180 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339183 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339185 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:26:48.339211 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339188 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339190 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339194 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339197 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339200 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339203 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339205 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339208 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339210 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339213 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339216 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339218 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339222 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339225 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339228 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339231 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339233 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339236 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339238 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:26:48.339884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339242 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339245 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339248 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339251 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339253 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339256 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339258 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339261 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339264 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339267 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339269 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339272 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339274 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339276 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339279 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339282 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339285 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339288 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339290 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339293 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:26:48.340572 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339295 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339298 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339301 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339303 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339306 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339308 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339311 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339315 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339318 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339321 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339323 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339326 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339329 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339331 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339334 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339336 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339339 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339341 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339344 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339346 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:26:48.341160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339349 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339352 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339355 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339357 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339360 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339362 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339365 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339369 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339371 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339374 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339376 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339379 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339381 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339384 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339386 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339389 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339391 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339394 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:26:48.341737 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.339396 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:26:48.342295 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342279 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:26:48.342295 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342296 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342300 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342304 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342306 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342309 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342312 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342314 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342317 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342320 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342322 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342325 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342328 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342330 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342333 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342336 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342339 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342341 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342344 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342346 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342349 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:26:48.342397 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342352 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342354 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342357 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342360 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342362 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342365 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342368 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342370 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342373 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342376 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342379 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342381 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342384 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342387 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342391 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342393 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342396 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342398 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342401 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:26:48.342913 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342405 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342407 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342410 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342413 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342415 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342418 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342420 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342423 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342425 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342428 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342433 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342438 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342440 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342443 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342446 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342448 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342451 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342453 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342456 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:26:48.343368 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342458 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342461 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342464 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342468 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342472 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342476 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342480 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342483 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342486 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342489 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342491 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342494 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342496 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342499 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342501 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342504 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342507 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342509 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342512 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:26:48.343845 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342515 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342517 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342520 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342523 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342525 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342528 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342531 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342533 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342536 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342539 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342541 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342544 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342546 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342549 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342552 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342554 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342557 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342559 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342562 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342565 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:26:48.344316 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342569 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342983 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342989 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342992 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342996 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.342998 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343002 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343005 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343008 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343011 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343014 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343016 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343019 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343022 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343025 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343028 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343030 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343033 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343036 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:26:48.344818 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343039 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343042 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343044 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343047 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343050 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343052 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343055 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343058 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343061 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343063 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343066 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343069 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343072 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343074 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343077 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343079 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343082 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343085 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343087 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343090 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:26:48.345300 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343100 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343104 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343106 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343109 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343112 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343115 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343117 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343120 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343122 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343125 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343128 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343131 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343134 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343136 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343140 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343143 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343145 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343148 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343151 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343154 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:26:48.345792 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343157 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343160 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343162 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343165 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343168 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343170 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343173 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343175 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343178 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343181 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343184 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343186 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343189 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343191 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343194 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343199 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343202 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343205 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343208 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:26:48.346270 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343211 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343213 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343216 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343219 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343221 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343225 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343228 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343231 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343234 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343236 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343239 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343242 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343245 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343249 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343252 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343255 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343259 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343263 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343266 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:26:48.346762 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343268 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343271 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343274 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343348 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343361 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343372 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343378 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343385 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343390 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343394 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343398 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343402 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343405 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343408 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343412 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343415 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343418 2557 flags.go:64] FLAG: --cgroup-root="" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343421 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343424 2557 flags.go:64] FLAG: --client-ca-file="" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343428 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343431 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343435 2557 flags.go:64] FLAG: --cluster-domain="" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343438 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 15:26:48.347240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343441 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343445 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343448 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343452 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343455 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343458 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343461 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343464 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343467 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343470 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343473 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343476 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343480 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343483 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343486 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343489 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343492 2557 flags.go:64] FLAG: --enable-server="true" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343495 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343500 2557 flags.go:64] FLAG: --event-burst="100" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343503 2557 flags.go:64] FLAG: --event-qps="50" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343506 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343509 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343512 2557 flags.go:64] FLAG: --eviction-hard="" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343515 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 15:26:48.347839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343518 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343521 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343524 2557 flags.go:64] FLAG: --eviction-soft="" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343527 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343530 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343534 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343537 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343540 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343543 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343546 2557 flags.go:64] FLAG: --feature-gates="" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343550 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343553 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343556 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343559 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343562 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343565 2557 flags.go:64] FLAG: --help="false" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343568 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343571 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343574 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343577 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343581 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343584 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343587 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 15:26:48.348427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343590 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343593 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343595 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343598 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343601 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343604 2557 flags.go:64] FLAG: --kube-reserved="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343607 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343610 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343612 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343616 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343618 2557 flags.go:64] FLAG: --lock-file="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343621 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343624 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343627 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343633 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343637 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343639 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343642 2557 flags.go:64] FLAG: --logging-format="text" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343645 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343649 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343651 2557 flags.go:64] FLAG: --manifest-url="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343654 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343658 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343661 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343665 2557 flags.go:64] FLAG: --max-pods="110" Jul 20 15:26:48.349038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343668 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343671 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343674 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343677 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343680 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343682 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343685 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343693 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343696 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343699 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343701 2557 flags.go:64] FLAG: --pod-cidr="" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343705 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343710 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343713 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343716 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343718 2557 flags.go:64] FLAG: --port="10250" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343721 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343724 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fd97226a56693432" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343728 2557 flags.go:64] FLAG: --qos-reserved="" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343731 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343734 2557 flags.go:64] FLAG: --register-node="true" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343736 2557 flags.go:64] FLAG: --register-with-taints="" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343740 2557 flags.go:64] FLAG: --registry-burst="10" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343743 2557 flags.go:64] FLAG: --registry-qps="5" Jul 20 15:26:48.349645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343746 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343749 2557 flags.go:64] FLAG: --reserved-memory="" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343753 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343756 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343758 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343761 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343764 2557 flags.go:64] FLAG: --runonce="false" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343781 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343784 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343787 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343790 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343793 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343796 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343799 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343802 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343804 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343807 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343810 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343813 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343816 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343819 2557 flags.go:64] FLAG: --system-cgroups="" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343821 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343827 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343829 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343832 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 15:26:48.350263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343836 2557 flags.go:64] FLAG: --tls-min-version="" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343838 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343841 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343848 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343851 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343854 2557 flags.go:64] FLAG: --v="2" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343859 2557 flags.go:64] FLAG: --version="false" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343864 2557 flags.go:64] FLAG: --vmodule="" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343867 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.343871 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343993 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.343997 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344000 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344003 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344006 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344008 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344011 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344014 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344016 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344019 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344022 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344024 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:26:48.350965 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344026 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344030 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344032 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344035 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344038 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344040 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344043 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344045 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344049 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344053 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344056 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344058 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344061 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344065 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344068 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344070 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344073 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344076 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344079 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:26:48.351536 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344082 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344085 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344087 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344090 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344094 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344097 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344100 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344103 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344106 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344109 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344112 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344115 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344118 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344121 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344124 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344127 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344130 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344134 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344137 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:26:48.352016 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344140 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344143 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344146 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344149 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344154 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344157 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344160 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344163 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344166 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344169 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344172 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344175 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344177 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344180 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344183 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344185 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344188 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344191 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344193 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344196 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:26:48.352475 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344199 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344201 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344204 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344206 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344209 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344212 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344214 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344217 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344220 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344222 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344225 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344227 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344230 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344232 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344235 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344237 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344241 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344244 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344247 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344250 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:26:48.352972 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344253 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344257 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344259 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344262 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344265 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344267 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344270 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344273 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.344276 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.344753 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.352451 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.352465 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.352483 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.352489 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352545 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:26:48.353455 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352549 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352554 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352558 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352561 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352565 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352568 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352570 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352573 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352575 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352578 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352581 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352583 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352586 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352589 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352591 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352594 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352596 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352599 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352601 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352604 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:26:48.353850 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352607 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352611 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352614 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352618 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352621 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352624 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352626 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352629 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352633 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352637 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352641 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352644 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352647 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352649 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352652 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352654 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352657 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352659 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352662 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:26:48.354330 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352664 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352667 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352670 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352672 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352675 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352677 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352680 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352682 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352685 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352687 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352690 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352693 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352695 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352698 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352700 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352702 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352705 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352708 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352711 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:26:48.354783 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352714 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352716 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352719 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352722 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352725 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352727 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352730 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352733 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352735 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352738 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352740 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352743 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352745 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352748 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352751 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352753 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352755 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352758 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352760 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352763 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:26:48.355284 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352766 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352783 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352786 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352788 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352791 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352794 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352796 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352799 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352801 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352804 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352806 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352809 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352812 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352814 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352817 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352819 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352823 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352826 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352829 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:26:48.355804 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352831 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.352837 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352956 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352961 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352964 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352967 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352970 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352972 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352975 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352978 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352980 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352983 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352986 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352989 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352992 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 15:26:48.356246 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352994 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.352997 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353000 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353002 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353004 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353007 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353009 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353012 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353016 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353019 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353022 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353025 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353028 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353031 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353033 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353037 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353039 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353042 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353044 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 15:26:48.356732 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353046 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353050 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353053 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353056 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353058 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353061 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353063 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353066 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353068 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353071 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353073 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353075 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353078 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353080 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353083 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353085 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353088 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353090 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353092 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353095 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 15:26:48.357286 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353098 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353100 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353102 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353105 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353107 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353110 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353113 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353115 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353118 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353121 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353124 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353126 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353129 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353131 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353134 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353136 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353139 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353142 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353145 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 15:26:48.358077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353147 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353150 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353152 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353155 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353157 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353160 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353162 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353164 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353167 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353169 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353172 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353174 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353176 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353179 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353182 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353184 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353187 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353189 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353192 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353194 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 15:26:48.358694 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353196 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353199 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353201 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353204 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353206 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353209 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353211 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:48.353214 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.353218 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.353326 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.355121 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.356329 2557 server.go:1019] "Starting client certificate rotation" Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.356444 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 15:26:48.359331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.356534 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 15:26:48.359654 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.358674 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 15:26:48.359840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.359823 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 15:26:48.373071 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.373048 2557 log.go:25] "Validated CRI v1 runtime API" Jul 20 15:26:48.375061 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.375046 2557 log.go:25] "Validated CRI v1 image API" Jul 20 15:26:48.375905 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.375889 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 15:26:48.379248 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.379229 2557 fs.go:135] Filesystem UUIDs: map[3161b29a-9ae2-4403-828c-7b0ead5563de:/dev/nvme0n1p4 63edb6c5-ba67-4af4-9ceb-79c52a5243dc:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 20 15:26:48.379314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.379247 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 15:26:48.385280 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.385176 2557 manager.go:216] Machine: {Timestamp:2026-07-20 15:26:48.383700157 +0000 UTC m=+0.328293517 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103332 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b3c56ab6499188a85445d88651031 SystemUUID:ec2b3c56-ab64-9918-8a85-445d88651031 BootID:4363a4b0-e72c-4616-826f-51555b14e4fe Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:db:ed:05:0f:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:db:ed:05:0f:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:59:aa:5a:96:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 15:26:48.385280 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.385275 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 15:26:48.385393 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.385382 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 15:26:48.388200 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.388176 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 15:26:48.388335 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.388201 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-150.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 15:26:48.388930 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.388921 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 15:26:48.388970 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.388932 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 15:26:48.388970 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.388940 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 15:26:48.389216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.389200 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 15:26:48.389562 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.389549 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 15:26:48.390793 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.390765 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 15:26:48.393719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393708 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 20 15:26:48.393858 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393848 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 15:26:48.393903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393879 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 20 15:26:48.393903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393889 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 15:26:48.393903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393903 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 15:26:48.394015 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393913 2557 kubelet.go:398] "Adding apiserver pod source" Jul 20 15:26:48.394015 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.393922 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 15:26:48.394265 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.394253 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 15:26:48.394300 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.394272 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 15:26:48.396607 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.396590 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 15:26:48.398374 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.398361 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 15:26:48.398424 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.398391 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 15:26:48.399399 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399383 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399405 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399411 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399416 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399425 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399430 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399438 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399443 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399449 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399454 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 15:26:48.399461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399462 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 15:26:48.399720 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.399468 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 15:26:48.400200 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.400189 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 15:26:48.400200 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.400198 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 15:26:48.403230 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.403208 2557 server.go:1268] "Started kubelet" Jul 20 15:26:48.403707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.403640 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 15:26:48.403884 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.403663 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 15:26:48.404126 ip-10-0-141-150 systemd[1]: Started Kubernetes Kubelet. Jul 20 15:26:48.407360 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.407318 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 15:26:48.408048 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.407983 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 15:26:48.408048 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.408028 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-150.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 15:26:48.408230 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.408208 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-150.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 15:26:48.409015 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.408970 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 15:26:48.411065 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.410648 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 20 15:26:48.413339 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.413320 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 15:26:48.414292 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.413377 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-150.ec2.internal.18c408f6cad04c55 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-150.ec2.internal,UID:ip-10-0-141-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-150.ec2.internal,},FirstTimestamp:2026-07-20 15:26:48.403184725 +0000 UTC m=+0.347778086,LastTimestamp:2026-07-20 15:26:48.403184725 +0000 UTC m=+0.347778086,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-150.ec2.internal,}" Jul 20 15:26:48.415826 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.415813 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 15:26:48.415895 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.415825 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 15:26:48.415967 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.415944 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.416318 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.416278 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 15:26:48.416387 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.416311 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 15:26:48.416387 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.416333 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 15:26:48.416461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.416428 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 20 15:26:48.416461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.416434 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 20 15:26:48.417996 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.417979 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 15:26:48.418060 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.417998 2557 factory.go:55] Registering systemd factory Jul 20 15:26:48.418060 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.418008 2557 factory.go:223] Registration of the systemd container factory successfully Jul 20 15:26:48.418347 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.418331 2557 factory.go:153] Registering CRI-O factory Jul 20 15:26:48.418347 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.418348 2557 factory.go:223] Registration of the crio container factory successfully Jul 20 15:26:48.418444 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.418371 2557 factory.go:103] Registering Raw factory Jul 20 15:26:48.418444 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.418385 2557 manager.go:1196] Started watching for new ooms in manager Jul 20 15:26:48.418788 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.418756 2557 manager.go:349] Starting recovery of all containers Jul 20 15:26:48.423474 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.423428 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-150.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 15:26:48.423573 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.423550 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 15:26:48.429554 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.429528 2557 manager.go:354] Recovery completed Jul 20 15:26:48.432721 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.432705 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5hsmj" Jul 20 15:26:48.434006 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.433995 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.436066 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436050 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.436135 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436084 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.436135 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436099 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.436683 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436670 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 15:26:48.436683 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436680 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 15:26:48.436840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436689 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.436840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.436716 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 20 15:26:48.437528 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.437511 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.437581 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.437543 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.437581 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.437559 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.438370 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.438345 2557 policy_none.go:49] "None policy: Start" Jul 20 15:26:48.438370 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.438363 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 15:26:48.438450 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.438374 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 15:26:48.438483 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.438361 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-150.ec2.internal.18c408f6ccc60fbe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-150.ec2.internal,UID:ip-10-0-141-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-150.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-150.ec2.internal,},FirstTimestamp:2026-07-20 15:26:48.436068286 +0000 UTC m=+0.380661647,LastTimestamp:2026-07-20 15:26:48.436068286 +0000 UTC m=+0.380661647,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-150.ec2.internal,}" Jul 20 15:26:48.439483 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.439463 2557 policy_none.go:47] "Start" Jul 20 15:26:48.443845 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.443828 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5hsmj" Jul 20 15:26:48.446046 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.445968 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-150.ec2.internal.18c408f6ccc66a87 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-150.ec2.internal,UID:ip-10-0-141-150.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-150.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-150.ec2.internal,},FirstTimestamp:2026-07-20 15:26:48.436091527 +0000 UTC m=+0.380684890,LastTimestamp:2026-07-20 15:26:48.436091527 +0000 UTC m=+0.380684890,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-150.ec2.internal,}" Jul 20 15:26:48.476356 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476337 2557 manager.go:337] "Starting Device Plugin manager" Jul 20 15:26:48.476452 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.476374 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 15:26:48.476452 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476383 2557 server.go:87] "Starting device plugin registration server" Jul 20 15:26:48.476641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476627 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 15:26:48.476689 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476643 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 15:26:48.476742 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476728 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 15:26:48.476840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476825 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 15:26:48.476889 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.476843 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 15:26:48.477355 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.477338 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 15:26:48.477431 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.477387 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.557664 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.557623 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 15:26:48.558988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.558970 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 15:26:48.559084 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.559010 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 15:26:48.559084 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.559033 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 15:26:48.559084 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.559067 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 15:26:48.561390 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.561374 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:26:48.577632 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.577613 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.578500 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.578480 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.578585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.578509 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.578585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.578520 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.578585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.578544 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.586799 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.586742 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.586799 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.586764 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-150.ec2.internal\": node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.600021 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.599989 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.659667 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.659636 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal"] Jul 20 15:26:48.659790 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.659705 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.660718 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.660703 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.660874 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.660731 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.660874 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.660741 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.662056 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662044 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.662173 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662160 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.662206 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662191 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.662783 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662751 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.662853 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662757 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.662853 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662818 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.662853 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662829 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.662938 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662799 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.662938 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.662872 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.664090 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.664076 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.664161 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.664099 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 15:26:48.664829 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.664813 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientMemory" Jul 20 15:26:48.664910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.664842 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 15:26:48.664910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.664858 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeHasSufficientPID" Jul 20 15:26:48.685411 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.685391 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-150.ec2.internal\" not found" node="ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.689927 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.689912 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-150.ec2.internal\" not found" node="ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.700999 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.700980 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.718027 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.718004 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2de5b1132291d4323877351deb872b0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal\" (UID: \"2de5b1132291d4323877351deb872b0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.718106 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.718029 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2de5b1132291d4323877351deb872b0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal\" (UID: \"2de5b1132291d4323877351deb872b0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.718106 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.718047 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/568e315d1498f6b05c03ff2bf976f5b2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-150.ec2.internal\" (UID: \"568e315d1498f6b05c03ff2bf976f5b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.801860 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.801834 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.818171 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.818146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2de5b1132291d4323877351deb872b0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal\" (UID: \"2de5b1132291d4323877351deb872b0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.818232 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.818184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2de5b1132291d4323877351deb872b0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal\" (UID: \"2de5b1132291d4323877351deb872b0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.818268 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.818239 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2de5b1132291d4323877351deb872b0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal\" (UID: \"2de5b1132291d4323877351deb872b0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.818309 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.818291 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2de5b1132291d4323877351deb872b0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal\" (UID: \"2de5b1132291d4323877351deb872b0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.818342 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.818314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/568e315d1498f6b05c03ff2bf976f5b2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-150.ec2.internal\" (UID: \"568e315d1498f6b05c03ff2bf976f5b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.818342 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.818339 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/568e315d1498f6b05c03ff2bf976f5b2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-150.ec2.internal\" (UID: \"568e315d1498f6b05c03ff2bf976f5b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.902673 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:48.902589 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:48.988137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.988105 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:48.992985 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:48.992759 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" Jul 20 15:26:49.003302 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.003164 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:49.103801 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.103750 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:49.204330 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.204253 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-150.ec2.internal\" not found" Jul 20 15:26:49.230161 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.230141 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:26:49.316430 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.316403 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" Jul 20 15:26:49.332888 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.332870 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 15:26:49.334471 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.334459 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" Jul 20 15:26:49.345438 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.345416 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 15:26:49.355550 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.355532 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 15:26:49.355661 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.355645 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 15:26:49.355699 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.355669 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 15:26:49.385024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.385000 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:26:49.394199 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.394175 2557 apiserver.go:52] "Watching apiserver" Jul 20 15:26:49.401617 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.401600 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 15:26:49.402585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.402565 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk","openshift-cluster-node-tuning-operator/tuned-nc8rl","openshift-dns/node-resolver-zkpd4","openshift-multus/multus-additional-cni-plugins-lmfv6","openshift-multus/network-metrics-daemon-gbvtz","openshift-ovn-kubernetes/ovnkube-node-6hzlp","kube-system/global-pull-secret-syncer-ddwq7","kube-system/konnectivity-agent-kq5qm","kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal","openshift-image-registry/node-ca-ntqhn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal","openshift-multus/multus-x2xtj","openshift-network-diagnostics/network-check-target-2bk9l","openshift-network-operator/iptables-alerter-fklcd"] Jul 20 15:26:49.403849 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.403831 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.403922 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.403898 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:26:49.405157 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.405143 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:49.405216 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.405199 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:26:49.406072 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.406058 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.407339 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.407317 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.408534 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.408429 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:49.408635 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.408543 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:26:49.409901 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.409882 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 15:26:49.410240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410228 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.410363 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410348 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.410414 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410397 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9wsjg\"" Jul 20 15:26:49.410598 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410581 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.410712 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410698 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bhg65\"" Jul 20 15:26:49.410875 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410861 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.410875 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410870 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 15:26:49.410985 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410919 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.410985 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.410933 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 15:26:49.411299 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.411285 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.412346 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.412329 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.413435 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.413420 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.414169 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414154 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.414329 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414313 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.414427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414334 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 15:26:49.414427 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414315 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.414532 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414449 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 15:26:49.414588 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414548 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 15:26:49.414634 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.414609 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.415879 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.415863 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 15:26:49.415929 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.415885 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 15:26:49.416057 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.416044 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.416257 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.416236 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.416369 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.416332 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dc2xv\"" Jul 20 15:26:49.416895 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.416882 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.416949 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.416929 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 15:26:49.416949 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.416929 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 15:26:49.417088 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417073 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.417220 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417204 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9phc4\"" Jul 20 15:26:49.417315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417299 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-569z8\"" Jul 20 15:26:49.417366 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417357 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.417417 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417376 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.417668 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417655 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.417910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.417892 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sqlwx\"" Jul 20 15:26:49.418033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.418020 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h9hh5\"" Jul 20 15:26:49.418093 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.418083 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 15:26:49.418932 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.418913 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 15:26:49.419988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.419973 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 15:26:49.420822 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420805 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb6z5\" (UniqueName: \"kubernetes.io/projected/b7662867-b0a8-4b7d-bb51-c6562a169daf-kube-api-access-nb6z5\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.420868 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420836 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-system-cni-dir\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.420868 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420854 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.420965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.420965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420931 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-modprobe-d\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.420965 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420951 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420965 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnzl\" (UniqueName: \"kubernetes.io/projected/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-kube-api-access-fmnzl\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.420998 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421005 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qv5vg\"" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-run-netns\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421030 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421045 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-ovn\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.421108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b748bbb0-366a-4a11-841d-0a899428492a-iptables-alerter-script\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421119 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/326a71ed-13ac-428f-8784-72a0bfb9eff2-tmp-dir\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421164 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chjgm\" (UniqueName: \"kubernetes.io/projected/326a71ed-13ac-428f-8784-72a0bfb9eff2-kube-api-access-chjgm\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421194 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-socket-dir-parent\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421223 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-k8s-cni-cncf-io\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421295 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-os-release\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-var-lib-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421346 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421362 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-log-socket\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.421389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovnkube-config\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421410 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-etc-selinux\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421442 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-etc-kubernetes\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b748bbb0-366a-4a11-841d-0a899428492a-host-slash\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421490 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mhxxr\"" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-run\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421533 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-var-lib-kubelet\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-device-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-serviceca\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421605 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-netns\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421632 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-node-log\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421652 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysconfig\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.421744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-systemd\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421746 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-host\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421794 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94dccc43-d5cf-4547-9ca7-b0122d3780f5-tmp\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/326a71ed-13ac-428f-8784-72a0bfb9eff2-hosts-file\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421831 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-system-cni-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-cni-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-os-release\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421877 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbsj\" (UniqueName: \"kubernetes.io/projected/94dccc43-d5cf-4547-9ca7-b0122d3780f5-kube-api-access-5jbsj\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421891 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421906 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-etc-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421930 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7662867-b0a8-4b7d-bb51-c6562a169daf-cni-binary-copy\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.421968 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-socket-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422001 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-multus-certs\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422026 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/598ac123-40fd-474c-b70e-8877d0aef642-kubelet-config\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422050 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pb2l\" (UniqueName: \"kubernetes.io/projected/515a8194-00d7-4d90-8091-44183e8d3bb4-kube-api-access-6pb2l\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422067 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-kubelet\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422111 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnq57\" (UniqueName: \"kubernetes.io/projected/b748bbb0-366a-4a11-841d-0a899428492a-kube-api-access-bnq57\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.422136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422141 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-hostroot\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422163 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-cnibin\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422187 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422209 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-cni-bin\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422281 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovn-node-metrics-cert\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422315 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-sys\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422339 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/598ac123-40fd-474c-b70e-8877d0aef642-dbus\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422354 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovnkube-script-lib\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422392 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysctl-conf\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422406 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-lib-modules\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422418 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-cnibin\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422432 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-sys-fs\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422463 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a13d15d-b2df-4eca-9a3f-f4ebdc229404-agent-certs\") pod \"konnectivity-agent-kq5qm\" (UID: \"2a13d15d-b2df-4eca-9a3f-f4ebdc229404\") " pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.422755 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422482 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a13d15d-b2df-4eca-9a3f-f4ebdc229404-konnectivity-ca\") pod \"konnectivity-agent-kq5qm\" (UID: \"2a13d15d-b2df-4eca-9a3f-f4ebdc229404\") " pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-daemon-config\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422511 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-env-overrides\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422526 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvhw\" (UniqueName: \"kubernetes.io/projected/d83e25c1-3be8-4655-ab70-513914c20325-kube-api-access-whvhw\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-systemd-units\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422567 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-registration-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422581 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9hz5\" (UniqueName: \"kubernetes.io/projected/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-kube-api-access-l9hz5\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422610 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysctl-d\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422627 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-cni-bin\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422640 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-cni-multus\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422668 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-kubelet\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422693 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-cni-netd\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4ts\" (UniqueName: \"kubernetes.io/projected/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-kube-api-access-jz4ts\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-kubernetes\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-tuned\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422787 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:49.423296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-host\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.423805 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422824 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-slash\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423805 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422853 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-systemd\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423805 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422871 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.423805 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422902 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.423805 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.422916 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-conf-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.432607 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.432588 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 15:26:49.447750 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.447726 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 15:21:49 +0000 UTC" deadline="2028-03-09 04:35:46.83344199 +0000 UTC" Jul 20 15:26:49.447750 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.447749 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14341h8m57.385695889s" Jul 20 15:26:49.473110 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.473088 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpfhn" Jul 20 15:26:49.485160 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.485133 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpfhn" Jul 20 15:26:49.517235 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.517214 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 15:26:49.523746 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523718 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94dccc43-d5cf-4547-9ca7-b0122d3780f5-tmp\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.523862 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/326a71ed-13ac-428f-8784-72a0bfb9eff2-hosts-file\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.523862 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-system-cni-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.523862 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-cni-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.523862 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523849 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/326a71ed-13ac-428f-8784-72a0bfb9eff2-hosts-file\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.524024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523883 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-os-release\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523907 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbsj\" (UniqueName: \"kubernetes.io/projected/94dccc43-d5cf-4547-9ca7-b0122d3780f5-kube-api-access-5jbsj\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.524024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523926 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-system-cni-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.524024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-etc-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.524024 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524014 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7662867-b0a8-4b7d-bb51-c6562a169daf-cni-binary-copy\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.523951 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-cni-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524038 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-socket-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524052 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524064 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-multus-certs\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/598ac123-40fd-474c-b70e-8877d0aef642-kubelet-config\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524093 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-os-release\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524114 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pb2l\" (UniqueName: \"kubernetes.io/projected/515a8194-00d7-4d90-8091-44183e8d3bb4-kube-api-access-6pb2l\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-etc-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524151 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-kubelet\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524146 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-socket-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524101 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524183 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/598ac123-40fd-474c-b70e-8877d0aef642-kubelet-config\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524144 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-multus-certs\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524191 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnq57\" (UniqueName: \"kubernetes.io/projected/b748bbb0-366a-4a11-841d-0a899428492a-kube-api-access-bnq57\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524232 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-hostroot\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-kubelet\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524258 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-cnibin\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.524314 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524291 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524310 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-hostroot\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524305 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-cnibin\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524344 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524369 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-cni-bin\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524392 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovn-node-metrics-cert\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-sys\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524435 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/598ac123-40fd-474c-b70e-8877d0aef642-dbus\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.524453 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524479 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovnkube-script-lib\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524503 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysctl-conf\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.524550 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:50.024497715 +0000 UTC m=+1.969091067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524592 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-lib-modules\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524619 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-cnibin\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524629 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-sys\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.525095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-sys-fs\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524671 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a13d15d-b2df-4eca-9a3f-f4ebdc229404-agent-certs\") pod \"konnectivity-agent-kq5qm\" (UID: \"2a13d15d-b2df-4eca-9a3f-f4ebdc229404\") " pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524688 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/598ac123-40fd-474c-b70e-8877d0aef642-dbus\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524695 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a13d15d-b2df-4eca-9a3f-f4ebdc229404-konnectivity-ca\") pod \"konnectivity-agent-kq5qm\" (UID: \"2a13d15d-b2df-4eca-9a3f-f4ebdc229404\") " pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524732 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524762 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-daemon-config\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524814 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7662867-b0a8-4b7d-bb51-c6562a169daf-cni-binary-copy\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524831 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-env-overrides\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524861 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvhw\" (UniqueName: \"kubernetes.io/projected/d83e25c1-3be8-4655-ab70-513914c20325-kube-api-access-whvhw\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524876 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524888 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-systemd-units\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524914 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524922 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-registration-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524925 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-cni-bin\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524928 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-systemd-units\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524593 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysctl-conf\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9hz5\" (UniqueName: \"kubernetes.io/projected/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-kube-api-access-l9hz5\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.526100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-registration-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.524979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysctl-d\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-cni-bin\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525034 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-cni-multus\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525058 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-kubelet\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-cni-netd\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525107 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4ts\" (UniqueName: \"kubernetes.io/projected/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-kube-api-access-jz4ts\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525131 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-kubernetes\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525154 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-tuned\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525235 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-host\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525258 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-slash\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-systemd\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525297 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-lib-modules\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525305 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525329 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525357 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-conf-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.526957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525364 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-env-overrides\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a13d15d-b2df-4eca-9a3f-f4ebdc229404-konnectivity-ca\") pod \"konnectivity-agent-kq5qm\" (UID: \"2a13d15d-b2df-4eca-9a3f-f4ebdc229404\") " pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525382 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb6z5\" (UniqueName: \"kubernetes.io/projected/b7662867-b0a8-4b7d-bb51-c6562a169daf-kube-api-access-nb6z5\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525417 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-system-cni-dir\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525449 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovnkube-script-lib\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525504 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-modprobe-d\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525522 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525607 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-cnibin\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525619 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-modprobe-d\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525679 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-sys-fs\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525720 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-host\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525724 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-system-cni-dir\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525726 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysctl-d\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525804 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-cni-bin\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525849 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-cni-multus\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-var-lib-kubelet\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.527798 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525940 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-cni-netd\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525955 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-slash\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525958 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-daemon-config\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525976 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnzl\" (UniqueName: \"kubernetes.io/projected/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-kube-api-access-fmnzl\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526008 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526012 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-systemd\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526034 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-run-netns\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526050 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-ovn\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526084 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d83e25c1-3be8-4655-ab70-513914c20325-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b748bbb0-366a-4a11-841d-0a899428492a-iptables-alerter-script\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-conf-dir\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526129 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/326a71ed-13ac-428f-8784-72a0bfb9eff2-tmp-dir\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526156 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chjgm\" (UniqueName: \"kubernetes.io/projected/326a71ed-13ac-428f-8784-72a0bfb9eff2-kube-api-access-chjgm\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-socket-dir-parent\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526196 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-host-run-netns\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526211 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-k8s-cni-cncf-io\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.528595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526238 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-os-release\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-var-lib-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526288 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-log-socket\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526312 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovnkube-config\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526336 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-etc-selinux\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526357 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-etc-kubernetes\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b748bbb0-366a-4a11-841d-0a899428492a-host-slash\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-run\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526437 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-var-lib-kubelet\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526462 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-device-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526491 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-serviceca\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526506 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-var-lib-openvswitch\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-netns\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.525447 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-kubernetes\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526546 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-node-log\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysconfig\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526596 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-systemd\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-host\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526641 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b748bbb0-366a-4a11-841d-0a899428492a-iptables-alerter-script\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-run-ovn\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526700 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-host\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526735 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526743 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-log-socket\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526788 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/326a71ed-13ac-428f-8784-72a0bfb9eff2-tmp-dir\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d83e25c1-3be8-4655-ab70-513914c20325-os-release\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.526464 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527266 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b748bbb0-366a-4a11-841d-0a899428492a-host-slash\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527030 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-node-log\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527168 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-sysconfig\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-netns\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527218 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovnkube-config\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.527300 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:50.027281444 +0000 UTC m=+1.971874793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527359 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-run\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527397 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-etc-selinux\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-systemd\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527464 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-etc-kubernetes\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.529729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.526990 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-multus-socket-dir-parent\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527477 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-serviceca\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527484 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-device-dir\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527537 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94dccc43-d5cf-4547-9ca7-b0122d3780f5-var-lib-kubelet\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.527597 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b7662867-b0a8-4b7d-bb51-c6562a169daf-host-run-k8s-cni-cncf-io\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.528313 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94dccc43-d5cf-4547-9ca7-b0122d3780f5-tmp\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.528392 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94dccc43-d5cf-4547-9ca7-b0122d3780f5-etc-tuned\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.528485 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-ovn-node-metrics-cert\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.530301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.528812 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a13d15d-b2df-4eca-9a3f-f4ebdc229404-agent-certs\") pod \"konnectivity-agent-kq5qm\" (UID: \"2a13d15d-b2df-4eca-9a3f-f4ebdc229404\") " pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.540526 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.540503 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4ts\" (UniqueName: \"kubernetes.io/projected/c9a8cd6d-0178-49fb-b191-0e7d148f7ea9-kube-api-access-jz4ts\") pod \"ovnkube-node-6hzlp\" (UID: \"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.540708 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.540571 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:26:49.540708 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.540588 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:26:49.540708 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.540598 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:49.540708 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.540611 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnq57\" (UniqueName: \"kubernetes.io/projected/b748bbb0-366a-4a11-841d-0a899428492a-kube-api-access-bnq57\") pod \"iptables-alerter-fklcd\" (UID: \"b748bbb0-366a-4a11-841d-0a899428492a\") " pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.540708 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:49.540644 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:50.040630602 +0000 UTC m=+1.985223969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:49.541158 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.541110 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbsj\" (UniqueName: \"kubernetes.io/projected/94dccc43-d5cf-4547-9ca7-b0122d3780f5-kube-api-access-5jbsj\") pod \"tuned-nc8rl\" (UID: \"94dccc43-d5cf-4547-9ca7-b0122d3780f5\") " pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.541266 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.541251 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pb2l\" (UniqueName: \"kubernetes.io/projected/515a8194-00d7-4d90-8091-44183e8d3bb4-kube-api-access-6pb2l\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:49.545520 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.545497 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnzl\" (UniqueName: \"kubernetes.io/projected/b0f588c0-c8fe-4221-9f83-4cc16b47d9da-kube-api-access-fmnzl\") pod \"aws-ebs-csi-driver-node-gpphk\" (UID: \"b0f588c0-c8fe-4221-9f83-4cc16b47d9da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.547325 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.547299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9hz5\" (UniqueName: \"kubernetes.io/projected/c42f52c3-250c-484e-a5bc-0b3c1c92f4dd-kube-api-access-l9hz5\") pod \"node-ca-ntqhn\" (UID: \"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd\") " pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.547610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.547589 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb6z5\" (UniqueName: \"kubernetes.io/projected/b7662867-b0a8-4b7d-bb51-c6562a169daf-kube-api-access-nb6z5\") pod \"multus-x2xtj\" (UID: \"b7662867-b0a8-4b7d-bb51-c6562a169daf\") " pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.547994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.547971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvhw\" (UniqueName: \"kubernetes.io/projected/d83e25c1-3be8-4655-ab70-513914c20325-kube-api-access-whvhw\") pod \"multus-additional-cni-plugins-lmfv6\" (UID: \"d83e25c1-3be8-4655-ab70-513914c20325\") " pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.548376 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.548357 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chjgm\" (UniqueName: \"kubernetes.io/projected/326a71ed-13ac-428f-8784-72a0bfb9eff2-kube-api-access-chjgm\") pod \"node-resolver-zkpd4\" (UID: \"326a71ed-13ac-428f-8784-72a0bfb9eff2\") " pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.560562 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.560539 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:26:49.570514 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.570495 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ntqhn" Jul 20 15:26:49.578867 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.578825 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:26:49.587065 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.587045 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x2xtj" Jul 20 15:26:49.659904 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.659861 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568e315d1498f6b05c03ff2bf976f5b2.slice/crio-afc88770fa16f6bb96e4f63b4c54fd295d34731e4d3a819cea0b8e7907bbd25d WatchSource:0}: Error finding container afc88770fa16f6bb96e4f63b4c54fd295d34731e4d3a819cea0b8e7907bbd25d: Status 404 returned error can't find the container with id afc88770fa16f6bb96e4f63b4c54fd295d34731e4d3a819cea0b8e7907bbd25d Jul 20 15:26:49.660255 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.660232 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de5b1132291d4323877351deb872b0a.slice/crio-2ff0b637497aaa36029e959224a0d7dfe8679b8dcb6541eb91cbe540d76d42f0 WatchSource:0}: Error finding container 2ff0b637497aaa36029e959224a0d7dfe8679b8dcb6541eb91cbe540d76d42f0: Status 404 returned error can't find the container with id 2ff0b637497aaa36029e959224a0d7dfe8679b8dcb6541eb91cbe540d76d42f0 Jul 20 15:26:49.664649 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.664633 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 15:26:49.744925 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.744833 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zkpd4" Jul 20 15:26:49.753423 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.753396 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod326a71ed_13ac_428f_8784_72a0bfb9eff2.slice/crio-33a8ccd8507b2d0a0ccc2425c80d430bd10718e3fb4e24ac1aa6f667ca8dbc85 WatchSource:0}: Error finding container 33a8ccd8507b2d0a0ccc2425c80d430bd10718e3fb4e24ac1aa6f667ca8dbc85: Status 404 returned error can't find the container with id 33a8ccd8507b2d0a0ccc2425c80d430bd10718e3fb4e24ac1aa6f667ca8dbc85 Jul 20 15:26:49.765332 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.765308 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" Jul 20 15:26:49.771657 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.771635 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:26:49.773661 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.773632 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83e25c1_3be8_4655_ab70_513914c20325.slice/crio-6e6c6547092bd135d0c984471c60614bf2f6685ceabdfd9d7414d32ac470913b WatchSource:0}: Error finding container 6e6c6547092bd135d0c984471c60614bf2f6685ceabdfd9d7414d32ac470913b: Status 404 returned error can't find the container with id 6e6c6547092bd135d0c984471c60614bf2f6685ceabdfd9d7414d32ac470913b Jul 20 15:26:49.779942 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.779911 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a8cd6d_0178_49fb_b191_0e7d148f7ea9.slice/crio-f846a4ebf440ffabb4c39302bbf01cba23170ac395ac00b70edcb00b8cbe9e49 WatchSource:0}: Error finding container f846a4ebf440ffabb4c39302bbf01cba23170ac395ac00b70edcb00b8cbe9e49: Status 404 returned error can't find the container with id f846a4ebf440ffabb4c39302bbf01cba23170ac395ac00b70edcb00b8cbe9e49 Jul 20 15:26:49.802601 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.802568 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fklcd" Jul 20 15:26:49.810887 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.810831 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb748bbb0_366a_4a11_841d_0a899428492a.slice/crio-2490b82bf444b7a89522b8205bef61ba0c822196e378e43045f786753f06159b WatchSource:0}: Error finding container 2490b82bf444b7a89522b8205bef61ba0c822196e378e43045f786753f06159b: Status 404 returned error can't find the container with id 2490b82bf444b7a89522b8205bef61ba0c822196e378e43045f786753f06159b Jul 20 15:26:49.816840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.816822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" Jul 20 15:26:49.824651 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.824628 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f588c0_c8fe_4221_9f83_4cc16b47d9da.slice/crio-b27833f43db569c953046b1d134915356a554b6a517b3e1496ca4e0ea256d25f WatchSource:0}: Error finding container b27833f43db569c953046b1d134915356a554b6a517b3e1496ca4e0ea256d25f: Status 404 returned error can't find the container with id b27833f43db569c953046b1d134915356a554b6a517b3e1496ca4e0ea256d25f Jul 20 15:26:49.839118 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:49.839092 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" Jul 20 15:26:49.849293 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.849267 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94dccc43_d5cf_4547_9ca7_b0122d3780f5.slice/crio-16330aa2a340103a7b541368f01e705a122d49780a9f9bb3be784cc0f0bcc412 WatchSource:0}: Error finding container 16330aa2a340103a7b541368f01e705a122d49780a9f9bb3be784cc0f0bcc412: Status 404 returned error can't find the container with id 16330aa2a340103a7b541368f01e705a122d49780a9f9bb3be784cc0f0bcc412 Jul 20 15:26:49.907051 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:49.907019 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7662867_b0a8_4b7d_bb51_c6562a169daf.slice/crio-5fb156ee1723bca23009b4866704eaea8a204a9786849bfc8cd5eed4fd359251 WatchSource:0}: Error finding container 5fb156ee1723bca23009b4866704eaea8a204a9786849bfc8cd5eed4fd359251: Status 404 returned error can't find the container with id 5fb156ee1723bca23009b4866704eaea8a204a9786849bfc8cd5eed4fd359251 Jul 20 15:26:50.013956 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:50.013926 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42f52c3_250c_484e_a5bc_0b3c1c92f4dd.slice/crio-c7c6355404aa03abeffac7be127c1fabac299323127e4eb9c4347d8905cdff50 WatchSource:0}: Error finding container c7c6355404aa03abeffac7be127c1fabac299323127e4eb9c4347d8905cdff50: Status 404 returned error can't find the container with id c7c6355404aa03abeffac7be127c1fabac299323127e4eb9c4347d8905cdff50 Jul 20 15:26:50.030140 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.030116 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:50.030234 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.030159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:50.030274 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.030254 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:50.030274 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.030255 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:50.030338 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.030306 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:51.030288796 +0000 UTC m=+2.974882165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:50.030338 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.030320 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:51.03031357 +0000 UTC m=+2.974906921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:50.130933 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.130899 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:50.131106 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.131090 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:26:50.131150 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.131111 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:26:50.131150 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.131121 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:50.131264 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.131179 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:51.131159618 +0000 UTC m=+3.075752991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:50.134314 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:26:50.134284 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a13d15d_b2df_4eca_9a3f_f4ebdc229404.slice/crio-e9b8444b125769f40834ec4900091f7357a7ef1c39949a4a96c657ab93ec9603 WatchSource:0}: Error finding container e9b8444b125769f40834ec4900091f7357a7ef1c39949a4a96c657ab93ec9603: Status 404 returned error can't find the container with id e9b8444b125769f40834ec4900091f7357a7ef1c39949a4a96c657ab93ec9603 Jul 20 15:26:50.461911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.461803 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:26:50.487159 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.487110 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:21:50 +0000 UTC" deadline="2028-02-10 12:07:18.056034172 +0000 UTC" Jul 20 15:26:50.487159 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.487158 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13676h40m27.568881857s" Jul 20 15:26:50.564093 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.564063 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:50.564270 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:50.564229 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:26:50.582369 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.582293 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x2xtj" event={"ID":"b7662867-b0a8-4b7d-bb51-c6562a169daf","Type":"ContainerStarted","Data":"5fb156ee1723bca23009b4866704eaea8a204a9786849bfc8cd5eed4fd359251"} Jul 20 15:26:50.591145 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.591102 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" event={"ID":"b0f588c0-c8fe-4221-9f83-4cc16b47d9da","Type":"ContainerStarted","Data":"b27833f43db569c953046b1d134915356a554b6a517b3e1496ca4e0ea256d25f"} Jul 20 15:26:50.597110 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.597074 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fklcd" event={"ID":"b748bbb0-366a-4a11-841d-0a899428492a","Type":"ContainerStarted","Data":"2490b82bf444b7a89522b8205bef61ba0c822196e378e43045f786753f06159b"} Jul 20 15:26:50.606141 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.606107 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"f846a4ebf440ffabb4c39302bbf01cba23170ac395ac00b70edcb00b8cbe9e49"} Jul 20 15:26:50.613726 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.613657 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerStarted","Data":"6e6c6547092bd135d0c984471c60614bf2f6685ceabdfd9d7414d32ac470913b"} Jul 20 15:26:50.617998 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.617912 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zkpd4" event={"ID":"326a71ed-13ac-428f-8784-72a0bfb9eff2","Type":"ContainerStarted","Data":"33a8ccd8507b2d0a0ccc2425c80d430bd10718e3fb4e24ac1aa6f667ca8dbc85"} Jul 20 15:26:50.624896 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.624865 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kq5qm" event={"ID":"2a13d15d-b2df-4eca-9a3f-f4ebdc229404","Type":"ContainerStarted","Data":"e9b8444b125769f40834ec4900091f7357a7ef1c39949a4a96c657ab93ec9603"} Jul 20 15:26:50.635100 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.635034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ntqhn" event={"ID":"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd","Type":"ContainerStarted","Data":"c7c6355404aa03abeffac7be127c1fabac299323127e4eb9c4347d8905cdff50"} Jul 20 15:26:50.638488 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.638435 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" event={"ID":"94dccc43-d5cf-4547-9ca7-b0122d3780f5","Type":"ContainerStarted","Data":"16330aa2a340103a7b541368f01e705a122d49780a9f9bb3be784cc0f0bcc412"} Jul 20 15:26:50.643864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.643825 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" event={"ID":"2de5b1132291d4323877351deb872b0a","Type":"ContainerStarted","Data":"2ff0b637497aaa36029e959224a0d7dfe8679b8dcb6541eb91cbe540d76d42f0"} Jul 20 15:26:50.653978 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:50.653946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" event={"ID":"568e315d1498f6b05c03ff2bf976f5b2","Type":"ContainerStarted","Data":"afc88770fa16f6bb96e4f63b4c54fd295d34731e4d3a819cea0b8e7907bbd25d"} Jul 20 15:26:51.038510 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.038459 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:51.038718 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.038562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:51.038718 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.038699 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:51.038718 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.038700 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:51.038966 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.038768 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:53.038748322 +0000 UTC m=+4.983341696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:51.038966 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.038788 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:53.038778369 +0000 UTC m=+4.983371725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:51.139969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.139829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:51.140155 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.140097 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:26:51.140155 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.140122 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:26:51.140155 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.140136 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:51.140348 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.140206 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:53.140186294 +0000 UTC m=+5.084779649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:51.488347 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.488272 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 15:21:50 +0000 UTC" deadline="2028-02-26 05:17:37.128877236 +0000 UTC" Jul 20 15:26:51.488347 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.488344 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14053h50m45.64053805s" Jul 20 15:26:51.559439 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.559403 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:51.559632 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.559551 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:26:51.560047 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:51.559994 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:51.560155 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:51.560133 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:26:52.214525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:52.214485 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 15:26:52.560095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:52.560020 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:52.560756 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:52.560178 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:26:53.057732 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:53.057632 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:53.057732 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:53.057714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:53.058092 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.057895 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:53.058092 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.057967 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:57.057947116 +0000 UTC m=+9.002540470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:53.058405 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.058385 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:53.058488 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.058443 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:57.058427091 +0000 UTC m=+9.003020446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:53.159073 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:53.159034 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:53.159315 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.159284 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:26:53.159398 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.159318 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:26:53.159398 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.159334 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:53.159514 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.159406 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:26:57.159385401 +0000 UTC m=+9.103978754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:53.559880 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:53.559837 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:53.560056 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.559981 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:26:53.560421 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:53.560386 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:53.560915 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:53.560482 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:26:54.562678 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:54.562157 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:54.562678 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:54.562304 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:26:55.559929 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:55.559869 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:55.560138 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:55.560012 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:26:55.560450 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:55.560427 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:55.560541 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:55.560529 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:26:56.559979 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:56.559384 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:56.559979 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:56.559546 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:26:57.096325 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:57.096277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:57.096515 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:57.096413 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:57.096515 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.096476 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:57.096618 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.096549 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:05.096531252 +0000 UTC m=+17.041124606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:26:57.096618 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.096563 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:57.096704 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.096622 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:05.096603694 +0000 UTC m=+17.041197069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:26:57.197318 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:57.197279 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:57.197504 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.197484 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:26:57.197566 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.197515 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:26:57.197566 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.197530 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:57.197694 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.197581 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:05.197566953 +0000 UTC m=+17.142160326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:26:57.559417 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:57.559330 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:57.559590 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:57.559331 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:57.559590 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.559483 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:26:57.559590 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:57.559542 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:26:58.561280 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:58.561240 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:26:58.561788 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:58.561382 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:26:59.559303 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:59.559270 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:26:59.559479 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:26:59.559272 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:26:59.559479 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:59.559398 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:26:59.559616 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:26:59.559509 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:00.562252 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:00.561665 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:00.562252 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:00.561833 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:01.559635 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:01.559594 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:01.559815 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:01.559747 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:01.560185 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:01.560169 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:01.560265 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:01.560250 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:02.559707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:02.559671 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:02.560107 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:02.559811 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:03.560071 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:03.560038 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:03.560558 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:03.560084 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:03.560558 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:03.560171 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:03.560558 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:03.560272 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:04.560006 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:04.559977 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:04.560185 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:04.560087 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:05.161960 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:05.161921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:05.162155 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:05.162009 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:05.162155 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.162141 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:27:05.162238 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.162201 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:21.162183264 +0000 UTC m=+33.106776612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:27:05.162411 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.162382 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:27:05.162508 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.162448 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:21.162429624 +0000 UTC m=+33.107022992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:27:05.262710 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:05.262672 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:05.262911 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.262878 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:27:05.262911 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.262900 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:27:05.263002 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.262914 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:27:05.263002 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.262975 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:21.262958311 +0000 UTC m=+33.207551675 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:27:05.560118 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:05.560030 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:05.560283 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.560162 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:05.560283 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:05.560218 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:05.560660 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:05.560324 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:06.559436 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:06.559402 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:06.559623 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:06.559532 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:07.559687 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:07.559645 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:07.560173 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:07.559705 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:07.560173 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:07.559852 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:07.560173 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:07.559936 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:08.560999 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:08.560968 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:08.561432 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:08.561094 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:09.559247 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:09.559212 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:09.559430 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:09.559258 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:09.559430 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:09.559333 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:09.559544 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:09.559424 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:10.559459 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:10.559429 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:10.560118 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:10.559554 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:11.559493 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.559225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:11.559971 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.559284 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:11.559971 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:11.559598 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:11.559971 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:11.559633 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:11.709264 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.709238 2557 generic.go:356] "Generic (PLEG): container finished" podID="2de5b1132291d4323877351deb872b0a" containerID="2a944dcf9df9f8aecfd4e782dfe1ad55b390e2d881b4b992822a1181654d067b" exitCode=0 Jul 20 15:27:11.709399 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.709319 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" event={"ID":"2de5b1132291d4323877351deb872b0a","Type":"ContainerDied","Data":"2a944dcf9df9f8aecfd4e782dfe1ad55b390e2d881b4b992822a1181654d067b"} Jul 20 15:27:11.710744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.710712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" event={"ID":"568e315d1498f6b05c03ff2bf976f5b2","Type":"ContainerStarted","Data":"65d41d59fc1226bca6140325a3eba29caf6f39cfc6d8ed2bbd2230f72d0c43d0"} Jul 20 15:27:11.712341 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.712318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x2xtj" event={"ID":"b7662867-b0a8-4b7d-bb51-c6562a169daf","Type":"ContainerStarted","Data":"61b834e2513bde347c5c5565c41216ea086dca5ebf6324442da6f489d6fb0db5"} Jul 20 15:27:11.713817 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.713798 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" event={"ID":"b0f588c0-c8fe-4221-9f83-4cc16b47d9da","Type":"ContainerStarted","Data":"1f26d0c38a2585cee4da5dfe7c130402853fdab5a11abe40ba1483c413727c29"} Jul 20 15:27:11.716575 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.716555 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:27:11.717261 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717136 2557 generic.go:356] "Generic (PLEG): container finished" podID="c9a8cd6d-0178-49fb-b191-0e7d148f7ea9" containerID="f34fa5aa6720ad4a93c935107ae8c5af665f9d0f48e2431ee62a3e6b9a417b52" exitCode=1 Jul 20 15:27:11.717261 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717201 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"922e007489395a7da66ce89e59471ae031651db0379b15dfa51c57b3b894ff77"} Jul 20 15:27:11.717380 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717269 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"575ced2566a254e8c27478b9ce95c49436398722e1e07d71d58e2831ea41de99"} Jul 20 15:27:11.717380 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717311 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"ebce85a578cd456a2f8bc97ac676da3b4bb43405e52f4da853a45e7e7b7bfae6"} Jul 20 15:27:11.717380 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717351 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"3418f88edf6c24f61d1d034d7522ac191d0d510784e5ced21637ed0c933d1764"} Jul 20 15:27:11.717519 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717389 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerDied","Data":"f34fa5aa6720ad4a93c935107ae8c5af665f9d0f48e2431ee62a3e6b9a417b52"} Jul 20 15:27:11.717519 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.717429 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"a28e4373f3a966bc37286b610382101ae2b482376b49a3f1cc07dc9c182f12ac"} Jul 20 15:27:11.719089 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.718878 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerStarted","Data":"9110553954b7dbf841d7ae52d8ce5a9780f3b56a9f2302ade32f108210d55607"} Jul 20 15:27:11.720446 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.720412 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zkpd4" event={"ID":"326a71ed-13ac-428f-8784-72a0bfb9eff2","Type":"ContainerStarted","Data":"bf74c110586f4d0bd92bc0792b19ac73c513a0766bcb21d2e8b983fb3e944bb6"} Jul 20 15:27:11.721762 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.721731 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kq5qm" event={"ID":"2a13d15d-b2df-4eca-9a3f-f4ebdc229404","Type":"ContainerStarted","Data":"266a124b6ff44ac5e5595e7cbf5edcc29a56d46e04ed2e61b8427267e4d49a27"} Jul 20 15:27:11.723251 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.723220 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ntqhn" event={"ID":"c42f52c3-250c-484e-a5bc-0b3c1c92f4dd","Type":"ContainerStarted","Data":"2bb2e2f0b7d909f292be64cf2ea29fcfa3a453ef2dc429aa138da100286a04dd"} Jul 20 15:27:11.724571 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.724551 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" event={"ID":"94dccc43-d5cf-4547-9ca7-b0122d3780f5","Type":"ContainerStarted","Data":"dc446a7318a78c964df7d52bfdd5272c0b9736d4be107a9aeb52e624d9a6688f"} Jul 20 15:27:11.754270 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.754228 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kq5qm" podStartSLOduration=1.904482285 podStartE2EDuration="22.754214422s" totalImagesPullingTime="20.849732137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:11.737577501 +0000 UTC m=+23.682170881" watchObservedRunningTime="2026-07-20 15:27:11.754214422 +0000 UTC m=+23.698807791" Jul 20 15:27:11.754396 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.754324 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ntqhn" podStartSLOduration=5.85684884 podStartE2EDuration="22.754318359s" totalImagesPullingTime="16.897469519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:11.754133804 +0000 UTC m=+23.698727175" watchObservedRunningTime="2026-07-20 15:27:11.754318359 +0000 UTC m=+23.698911728" Jul 20 15:27:11.776409 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.776370 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x2xtj" podStartSLOduration=1.6516014700000001 podStartE2EDuration="22.776356275s" totalImagesPullingTime="21.124754805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:11.775724378 +0000 UTC m=+23.720317752" watchObservedRunningTime="2026-07-20 15:27:11.776356275 +0000 UTC m=+23.720949644" Jul 20 15:27:11.790231 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.790196 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-150.ec2.internal" podStartSLOduration=21.790182058 podStartE2EDuration="21.790182058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:11.789678313 +0000 UTC m=+23.734271686" watchObservedRunningTime="2026-07-20 15:27:11.790182058 +0000 UTC m=+23.734775429" Jul 20 15:27:11.803497 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.803463 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zkpd4" podStartSLOduration=1.573099969 podStartE2EDuration="22.803451572s" totalImagesPullingTime="21.230351603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:11.803080798 +0000 UTC m=+23.747674169" watchObservedRunningTime="2026-07-20 15:27:11.803451572 +0000 UTC m=+23.748044942" Jul 20 15:27:11.841546 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:11.841512 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nc8rl" podStartSLOduration=1.706745748 podStartE2EDuration="22.841499423s" totalImagesPullingTime="21.134753675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:11.841014928 +0000 UTC m=+23.785608299" watchObservedRunningTime="2026-07-20 15:27:11.841499423 +0000 UTC m=+23.786092793" Jul 20 15:27:12.560416 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.560245 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:12.561006 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:12.560497 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:12.727191 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.727164 2557 generic.go:356] "Generic (PLEG): container finished" podID="d83e25c1-3be8-4655-ab70-513914c20325" containerID="9110553954b7dbf841d7ae52d8ce5a9780f3b56a9f2302ade32f108210d55607" exitCode=0 Jul 20 15:27:12.727308 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.727208 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerDied","Data":"9110553954b7dbf841d7ae52d8ce5a9780f3b56a9f2302ade32f108210d55607"} Jul 20 15:27:12.729035 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.729006 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" event={"ID":"2de5b1132291d4323877351deb872b0a","Type":"ContainerStarted","Data":"112fd7a04f40c9a72376e9b8353dfbccf0e4ec3a767d5ea20b530b852cdbd1cf"} Jul 20 15:27:12.730331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.730304 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fklcd" event={"ID":"b748bbb0-366a-4a11-841d-0a899428492a","Type":"ContainerStarted","Data":"584d1af2882eceb4d1d4c32cc184e255c7da558953abcd846701c5340e17e5b3"} Jul 20 15:27:12.759402 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.759367 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-150.ec2.internal" podStartSLOduration=22.75935555 podStartE2EDuration="22.75935555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:12.759256315 +0000 UTC m=+24.703849682" watchObservedRunningTime="2026-07-20 15:27:12.75935555 +0000 UTC m=+24.703948920" Jul 20 15:27:12.771556 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:12.771521 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fklcd" podStartSLOduration=6.670693691 podStartE2EDuration="23.771509059s" totalImagesPullingTime="17.100815368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:12.770933564 +0000 UTC m=+24.715526934" watchObservedRunningTime="2026-07-20 15:27:12.771509059 +0000 UTC m=+24.716102428" Jul 20 15:27:13.041258 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.041229 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 15:27:13.496818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.496709 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T15:27:13.041254779Z","UUID":"289519ce-00dc-478a-ae4f-f64156710319","Handler":null,"Name":"","Endpoint":""} Jul 20 15:27:13.498288 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.498266 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 15:27:13.498410 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.498310 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 15:27:13.560198 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.560169 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:13.560341 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:13.560275 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:13.560411 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.560337 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:13.560467 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:13.560449 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:13.735289 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:13.735240 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" event={"ID":"b0f588c0-c8fe-4221-9f83-4cc16b47d9da","Type":"ContainerStarted","Data":"441515ad54a5a545b9ddd99443a973cf17fb927856242eeb656c20178f37d1bc"} Jul 20 15:27:14.560251 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:14.560225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:14.560376 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:14.560342 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:14.739129 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:14.739055 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" event={"ID":"b0f588c0-c8fe-4221-9f83-4cc16b47d9da","Type":"ContainerStarted","Data":"8ad8897d837c16a69b1aed2b0478305feadc68b98b8a724b7ac125a43be9cd9d"} Jul 20 15:27:14.741958 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:14.741940 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:27:14.742325 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:14.742306 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"66a4dcc3b4769b4a41dbca504fe04b2a6b8988f902eaee8cbf652a5419b8c208"} Jul 20 15:27:14.758236 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:14.758178 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpphk" podStartSLOduration=1.908946776 podStartE2EDuration="25.758161578s" totalImagesPullingTime="23.849214802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:14.757592882 +0000 UTC m=+26.702186252" watchObservedRunningTime="2026-07-20 15:27:14.758161578 +0000 UTC m=+26.702754950" Jul 20 15:27:15.028560 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:15.028481 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:27:15.029485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:15.029468 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:27:15.559486 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:15.559458 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:15.559677 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:15.559578 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:15.559845 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:15.559818 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:15.559989 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:15.559931 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:15.745241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:15.745025 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:27:15.745876 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:15.745855 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kq5qm" Jul 20 15:27:16.560181 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:16.560151 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:16.560340 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:16.560297 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:17.559691 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:17.559664 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:17.559691 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:17.559679 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:17.560118 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:17.559816 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:17.560118 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:17.559942 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:17.752481 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:17.752417 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:27:17.752821 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:17.752802 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"6d5201842b1ab07a4e7e301213d9ee33ec0c1d04ad39f7244f324425520e42e1"} Jul 20 15:27:17.753354 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:17.753335 2557 scope.go:117] "RemoveContainer" containerID="f34fa5aa6720ad4a93c935107ae8c5af665f9d0f48e2431ee62a3e6b9a417b52" Jul 20 15:27:18.560455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.560421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:18.561059 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:18.560513 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:18.759047 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.758983 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:27:18.759363 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.759337 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" event={"ID":"c9a8cd6d-0178-49fb-b191-0e7d148f7ea9","Type":"ContainerStarted","Data":"ba409c76b83e99b74f5be6d3ec2888dc3518d8c2cb3acd33675bca965277ea84"} Jul 20 15:27:18.759540 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.759505 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:27:18.759672 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.759585 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:27:18.759672 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.759633 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:27:18.761438 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.761413 2557 generic.go:356] "Generic (PLEG): container finished" podID="d83e25c1-3be8-4655-ab70-513914c20325" containerID="eb04db6eda6cb48bf59ec786fef3778e05ec8a6b162f4699a5357d138ee6daf3" exitCode=0 Jul 20 15:27:18.761558 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.761443 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerDied","Data":"eb04db6eda6cb48bf59ec786fef3778e05ec8a6b162f4699a5357d138ee6daf3"} Jul 20 15:27:18.775710 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.775686 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:27:18.775879 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.775865 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:27:18.784739 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:18.784702 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" podStartSLOduration=8.524758352 podStartE2EDuration="29.784690316s" totalImagesPullingTime="21.259931964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:18.783306916 +0000 UTC m=+30.727900286" watchObservedRunningTime="2026-07-20 15:27:18.784690316 +0000 UTC m=+30.729283700" Jul 20 15:27:19.559396 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:19.559359 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:19.559610 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:19.559478 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:19.559610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:19.559485 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:19.559610 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:19.559589 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:20.560252 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:20.560218 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:20.560756 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:20.560328 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:20.767634 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:20.767595 2557 generic.go:356] "Generic (PLEG): container finished" podID="d83e25c1-3be8-4655-ab70-513914c20325" containerID="904e5698864584d652c92c3206934e289a56245e8948f3edcf6c144e2fc803cc" exitCode=0 Jul 20 15:27:20.767802 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:20.767671 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerDied","Data":"904e5698864584d652c92c3206934e289a56245e8948f3edcf6c144e2fc803cc"} Jul 20 15:27:21.177518 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:21.177476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:21.177713 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:21.177547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:21.177713 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.177637 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 15:27:21.177713 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.177657 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:27:21.177860 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.177717 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret podName:598ac123-40fd-474c-b70e-8877d0aef642 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:53.177699002 +0000 UTC m=+65.122292374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret") pod "global-pull-secret-syncer-ddwq7" (UID: "598ac123-40fd-474c-b70e-8877d0aef642") : object "kube-system"/"original-pull-secret" not registered Jul 20 15:27:21.177860 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.177733 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs podName:515a8194-00d7-4d90-8091-44183e8d3bb4 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:53.177726532 +0000 UTC m=+65.122319881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs") pod "network-metrics-daemon-gbvtz" (UID: "515a8194-00d7-4d90-8091-44183e8d3bb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 15:27:21.278611 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:21.278575 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:21.278784 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.278751 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 15:27:21.278852 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.278790 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 15:27:21.278852 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.278801 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5c27t for pod openshift-network-diagnostics/network-check-target-2bk9l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:27:21.278930 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.278858 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t podName:65630c99-f328-4ee7-96c6-7b4b282c1552 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:53.278842767 +0000 UTC m=+65.223436117 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5c27t" (UniqueName: "kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t") pod "network-check-target-2bk9l" (UID: "65630c99-f328-4ee7-96c6-7b4b282c1552") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 15:27:21.559921 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:21.559848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:21.560055 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:21.559848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:21.560055 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.559964 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:21.560055 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:21.560021 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:22.559707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:22.559674 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:22.560096 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:22.559819 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:22.774749 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:22.774715 2557 generic.go:356] "Generic (PLEG): container finished" podID="d83e25c1-3be8-4655-ab70-513914c20325" containerID="0f7e0176063a577e3e7ea9c3e87056afcaa1578c626073052713d67ae158d2ca" exitCode=0 Jul 20 15:27:22.774917 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:22.774765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerDied","Data":"0f7e0176063a577e3e7ea9c3e87056afcaa1578c626073052713d67ae158d2ca"} Jul 20 15:27:23.559345 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:23.559313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:23.559524 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:23.559315 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:23.559524 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:23.559419 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:23.559609 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:23.559507 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:24.560239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:24.560203 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:24.560676 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:24.560360 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:25.559412 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:25.559376 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:25.559595 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:25.559502 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:25.559595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:25.559559 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:25.559721 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:25.559675 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:26.559390 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:26.559357 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:26.559869 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:26.559499 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:27.559728 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:27.559680 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:27.559728 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:27.559730 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:27.560260 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:27.559799 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:27.560260 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:27.559922 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:27.789182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:27.789111 2557 generic.go:356] "Generic (PLEG): container finished" podID="d83e25c1-3be8-4655-ab70-513914c20325" containerID="f3ac1fb0e991743549ce9244c813c9c140bdbaba8f9b64a38ce92fc9eb45a49c" exitCode=0 Jul 20 15:27:27.789182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:27.789149 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerDied","Data":"f3ac1fb0e991743549ce9244c813c9c140bdbaba8f9b64a38ce92fc9eb45a49c"} Jul 20 15:27:28.560397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:28.560241 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:28.560723 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:28.560451 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:28.794497 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:28.794465 2557 generic.go:356] "Generic (PLEG): container finished" podID="d83e25c1-3be8-4655-ab70-513914c20325" containerID="085aa8ca1770e00fb722a62ee70dab80b29c89fe93490705cebf5fbfb02161cb" exitCode=0 Jul 20 15:27:28.794661 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:28.794519 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerDied","Data":"085aa8ca1770e00fb722a62ee70dab80b29c89fe93490705cebf5fbfb02161cb"} Jul 20 15:27:29.560159 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:29.560130 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:29.560345 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:29.560220 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:29.560345 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:29.560275 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:29.560409 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:29.560352 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:29.803606 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:29.803566 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" event={"ID":"d83e25c1-3be8-4655-ab70-513914c20325","Type":"ContainerStarted","Data":"f7411632af479345ea32750d92fc3ccee151aa32329129b7f621d52f2a50c295"} Jul 20 15:27:29.894710 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:29.894665 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lmfv6" podStartSLOduration=7.688146576 podStartE2EDuration="40.894649054s" totalImagesPullingTime="33.206502478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:29.876511637 +0000 UTC m=+41.821105007" watchObservedRunningTime="2026-07-20 15:27:29.894649054 +0000 UTC m=+41.839242415" Jul 20 15:27:30.559780 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:30.559737 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:30.559919 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:30.559860 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:31.560241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.560213 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:31.560569 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:31.560339 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:31.560569 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.560392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:31.560569 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:31.560505 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:31.672033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.672007 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddwq7"] Jul 20 15:27:31.675529 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.675503 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bk9l"] Jul 20 15:27:31.676201 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.676176 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gbvtz"] Jul 20 15:27:31.676305 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.676279 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:31.676563 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:31.676452 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:31.810221 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.810162 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:31.810221 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:31.810194 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:31.810390 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:31.810274 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:31.810390 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:31.810379 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:33.559816 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:33.559764 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:33.559816 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:33.559812 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:33.560221 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:33.559888 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:33.560221 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:33.559892 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:33.560221 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:33.559968 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:33.560221 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:33.560050 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:35.559402 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.559362 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:35.559402 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.559395 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:35.559793 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:35.559481 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2bk9l" podUID="65630c99-f328-4ee7-96c6-7b4b282c1552" Jul 20 15:27:35.559793 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.559508 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:35.559793 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:35.559571 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddwq7" podUID="598ac123-40fd-474c-b70e-8877d0aef642" Jul 20 15:27:35.559793 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:35.559630 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gbvtz" podUID="515a8194-00d7-4d90-8091-44183e8d3bb4" Jul 20 15:27:35.910275 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.910245 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-150.ec2.internal" event="NodeReady" Jul 20 15:27:35.910458 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.910384 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 15:27:35.959538 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.959511 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2c5dx"] Jul 20 15:27:35.976474 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.976439 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2c5dx"] Jul 20 15:27:35.976625 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.976575 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:35.977162 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.977145 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lhfxq"] Jul 20 15:27:35.978675 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.978654 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 15:27:35.978909 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.978895 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xj82\"" Jul 20 15:27:35.978999 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.978940 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 15:27:35.987906 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.987891 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:35.989269 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.989248 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lhfxq"] Jul 20 15:27:35.989853 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.989835 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 15:27:35.990291 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.990273 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 15:27:35.990373 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.990278 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qzpxj\"" Jul 20 15:27:35.990589 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:35.990570 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 15:27:36.094332 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.094306 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxnl7\" (UniqueName: \"kubernetes.io/projected/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-kube-api-access-pxnl7\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.094451 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.094335 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-metrics-tls\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.094451 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.094351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bfe5645-2124-423a-921e-f294097db8d9-cert\") pod \"ingress-canary-lhfxq\" (UID: \"5bfe5645-2124-423a-921e-f294097db8d9\") " pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.094451 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.094368 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-config-volume\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.094451 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.094428 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdqwv\" (UniqueName: \"kubernetes.io/projected/5bfe5645-2124-423a-921e-f294097db8d9-kube-api-access-sdqwv\") pod \"ingress-canary-lhfxq\" (UID: \"5bfe5645-2124-423a-921e-f294097db8d9\") " pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.094637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.094496 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-tmp-dir\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.195662 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.195598 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-config-volume\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.195662 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.195633 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdqwv\" (UniqueName: \"kubernetes.io/projected/5bfe5645-2124-423a-921e-f294097db8d9-kube-api-access-sdqwv\") pod \"ingress-canary-lhfxq\" (UID: \"5bfe5645-2124-423a-921e-f294097db8d9\") " pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.195864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.195679 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-tmp-dir\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.195864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.195763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnl7\" (UniqueName: \"kubernetes.io/projected/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-kube-api-access-pxnl7\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.195864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.195813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-metrics-tls\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.195864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.195836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bfe5645-2124-423a-921e-f294097db8d9-cert\") pod \"ingress-canary-lhfxq\" (UID: \"5bfe5645-2124-423a-921e-f294097db8d9\") " pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.196391 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.196250 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-tmp-dir\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.199315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.196712 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-config-volume\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.201112 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.201090 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bfe5645-2124-423a-921e-f294097db8d9-cert\") pod \"ingress-canary-lhfxq\" (UID: \"5bfe5645-2124-423a-921e-f294097db8d9\") " pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.201196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.201182 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-metrics-tls\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.203640 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.203614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxnl7\" (UniqueName: \"kubernetes.io/projected/df2e94b7-bd9a-479c-9472-a69cb1bf9e58-kube-api-access-pxnl7\") pod \"dns-default-2c5dx\" (UID: \"df2e94b7-bd9a-479c-9472-a69cb1bf9e58\") " pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.203866 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.203851 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdqwv\" (UniqueName: \"kubernetes.io/projected/5bfe5645-2124-423a-921e-f294097db8d9-kube-api-access-sdqwv\") pod \"ingress-canary-lhfxq\" (UID: \"5bfe5645-2124-423a-921e-f294097db8d9\") " pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.285815 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.285766 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:36.296881 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.296860 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lhfxq" Jul 20 15:27:36.528644 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.528615 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2c5dx"] Jul 20 15:27:36.531392 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.531210 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lhfxq"] Jul 20 15:27:36.534469 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:36.534443 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2e94b7_bd9a_479c_9472_a69cb1bf9e58.slice/crio-d167675b8a6ac7c741497f72fa5e4c9fed22935a6c2f99ca801396ae6ffbbf95 WatchSource:0}: Error finding container d167675b8a6ac7c741497f72fa5e4c9fed22935a6c2f99ca801396ae6ffbbf95: Status 404 returned error can't find the container with id d167675b8a6ac7c741497f72fa5e4c9fed22935a6c2f99ca801396ae6ffbbf95 Jul 20 15:27:36.534697 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:36.534674 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfe5645_2124_423a_921e_f294097db8d9.slice/crio-60aae2fa7ea7fde88111e5641a8deec4eb4520e8d20f700a5c4cab9f0891724d WatchSource:0}: Error finding container 60aae2fa7ea7fde88111e5641a8deec4eb4520e8d20f700a5c4cab9f0891724d: Status 404 returned error can't find the container with id 60aae2fa7ea7fde88111e5641a8deec4eb4520e8d20f700a5c4cab9f0891724d Jul 20 15:27:36.821602 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.821528 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2c5dx" event={"ID":"df2e94b7-bd9a-479c-9472-a69cb1bf9e58","Type":"ContainerStarted","Data":"d167675b8a6ac7c741497f72fa5e4c9fed22935a6c2f99ca801396ae6ffbbf95"} Jul 20 15:27:36.822456 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:36.822428 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lhfxq" event={"ID":"5bfe5645-2124-423a-921e-f294097db8d9","Type":"ContainerStarted","Data":"60aae2fa7ea7fde88111e5641a8deec4eb4520e8d20f700a5c4cab9f0891724d"} Jul 20 15:27:37.559263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.559228 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:37.559724 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.559692 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:37.560163 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.560145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:37.564820 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.564798 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 15:27:37.565038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.565022 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 15:27:37.565155 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.565130 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 15:27:37.565406 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.565391 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zgzvz\"" Jul 20 15:27:37.565593 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.565581 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tnhrs\"" Jul 20 15:27:37.567284 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:37.567105 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 15:27:39.816850 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:39.816687 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw"] Jul 20 15:27:39.846051 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:39.846031 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw"] Jul 20 15:27:39.846148 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:39.846124 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:39.848116 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:39.848094 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 20 15:27:39.848275 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:39.848094 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g4r5w\"" Jul 20 15:27:40.028969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.028940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-grmsw\" (UID: \"6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:40.129853 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.129812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-grmsw\" (UID: \"6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:40.130053 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:40.129989 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 20 15:27:40.130124 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:40.130072 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2-tls-certificates podName:6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:40.630052176 +0000 UTC m=+52.574645524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-grmsw" (UID: "6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2") : secret "prometheus-operator-admission-webhook-tls" not found Jul 20 15:27:40.476188 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.476152 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nghgd"] Jul 20 15:27:40.498909 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.498885 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nghgd"] Jul 20 15:27:40.499034 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.498995 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.502099 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.502068 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 15:27:40.502099 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.502079 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 15:27:40.502249 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.502116 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 15:27:40.502363 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.502349 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 15:27:40.502417 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.502402 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9rlnk\"" Jul 20 15:27:40.634313 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.634287 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-grmsw\" (UID: \"6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:40.634525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.634326 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/493dd2d2-5ce0-417c-8598-444afa797b51-crio-socket\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.634525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.634346 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/493dd2d2-5ce0-417c-8598-444afa797b51-data-volume\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.634525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.634362 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.634525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.634450 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.634525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.634516 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd58f\" (UniqueName: \"kubernetes.io/projected/493dd2d2-5ce0-417c-8598-444afa797b51-kube-api-access-gd58f\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.638023 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.638005 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-grmsw\" (UID: \"6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:40.734988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.734935 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/493dd2d2-5ce0-417c-8598-444afa797b51-crio-socket\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.734988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.734966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/493dd2d2-5ce0-417c-8598-444afa797b51-data-volume\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.734988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.734984 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.735163 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.735011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.735163 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.735056 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd58f\" (UniqueName: \"kubernetes.io/projected/493dd2d2-5ce0-417c-8598-444afa797b51-kube-api-access-gd58f\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.735163 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:40.735090 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 15:27:40.735163 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:40.735135 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 15:27:40.735163 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:40.735152 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:41.235138195 +0000 UTC m=+53.179731543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : configmap "kube-rbac-proxy" not found Jul 20 15:27:40.735360 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.735091 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/493dd2d2-5ce0-417c-8598-444afa797b51-crio-socket\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.735360 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:40.735184 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:41.235168774 +0000 UTC m=+53.179762122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : secret "insights-runtime-extractor-tls" not found Jul 20 15:27:40.735360 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.735345 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/493dd2d2-5ce0-417c-8598-444afa797b51-data-volume\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.748187 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.748162 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd58f\" (UniqueName: \"kubernetes.io/projected/493dd2d2-5ce0-417c-8598-444afa797b51-kube-api-access-gd58f\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:40.755424 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.755407 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:40.833604 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.833572 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lhfxq" event={"ID":"5bfe5645-2124-423a-921e-f294097db8d9","Type":"ContainerStarted","Data":"ad01eaba606d3214a809123735d9f70c4145128bd59cacf5b654a650754da22f"} Jul 20 15:27:40.835972 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.835950 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2c5dx" event={"ID":"df2e94b7-bd9a-479c-9472-a69cb1bf9e58","Type":"ContainerStarted","Data":"a61d805026a532411f9a9b97000d6c47b34c81eaf1abbe920c3c53eca722ede2"} Jul 20 15:27:40.836069 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.835989 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2c5dx" event={"ID":"df2e94b7-bd9a-479c-9472-a69cb1bf9e58","Type":"ContainerStarted","Data":"bd9e5b4bd1510fb37bf7608b0792e33b98ccf5a76c4a0b2c0551c92ec22bb238"} Jul 20 15:27:40.836142 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.836086 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:40.853985 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.852309 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lhfxq" podStartSLOduration=2.673486134 podStartE2EDuration="5.852288545s" totalImagesPullingTime="3.178802411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:40.849457762 +0000 UTC m=+52.794051132" watchObservedRunningTime="2026-07-20 15:27:40.852288545 +0000 UTC m=+52.796881918" Jul 20 15:27:40.869035 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.868985 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2c5dx" podStartSLOduration=2.6961306 podStartE2EDuration="5.868968425s" totalImagesPullingTime="3.172837825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:40.86881877 +0000 UTC m=+52.813412141" watchObservedRunningTime="2026-07-20 15:27:40.868968425 +0000 UTC m=+52.813561796" Jul 20 15:27:40.890257 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:40.890228 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw"] Jul 20 15:27:40.895604 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:40.895576 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9adadb_cd8d_404b_9f8a_4eddbe6b98c2.slice/crio-f67d942563a211d4ad4372723624276c7a3651f5eb727ce2ae694c351eee5e3c WatchSource:0}: Error finding container f67d942563a211d4ad4372723624276c7a3651f5eb727ce2ae694c351eee5e3c: Status 404 returned error can't find the container with id f67d942563a211d4ad4372723624276c7a3651f5eb727ce2ae694c351eee5e3c Jul 20 15:27:41.239066 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:41.238891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:41.239231 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:41.239086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:41.239231 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:41.239011 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 15:27:41.239231 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:41.239192 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:42.239175991 +0000 UTC m=+54.183769343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : configmap "kube-rbac-proxy" not found Jul 20 15:27:41.239231 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:41.239208 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 15:27:41.239373 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:41.239247 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:42.239236496 +0000 UTC m=+54.183829844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : secret "insights-runtime-extractor-tls" not found Jul 20 15:27:41.498094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:41.498021 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2c5dx_df2e94b7-bd9a-479c-9472-a69cb1bf9e58/dns/0.log" Jul 20 15:27:41.669997 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:41.669971 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2c5dx_df2e94b7-bd9a-479c-9472-a69cb1bf9e58/kube-rbac-proxy/0.log" Jul 20 15:27:41.840252 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:41.840163 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" event={"ID":"6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2","Type":"ContainerStarted","Data":"f67d942563a211d4ad4372723624276c7a3651f5eb727ce2ae694c351eee5e3c"} Jul 20 15:27:42.247073 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.247039 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:42.247227 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.247120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:42.247227 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:42.247179 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 15:27:42.247227 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:42.247195 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 15:27:42.247320 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:42.247246 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:44.24723136 +0000 UTC m=+56.191824708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : secret "insights-runtime-extractor-tls" not found Jul 20 15:27:42.247320 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:42.247259 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:44.247253317 +0000 UTC m=+56.191846665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : configmap "kube-rbac-proxy" not found Jul 20 15:27:42.669148 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.669128 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zkpd4_326a71ed-13ac-428f-8784-72a0bfb9eff2/dns-node-resolver/0.log" Jul 20 15:27:42.843586 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.843558 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" event={"ID":"6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2","Type":"ContainerStarted","Data":"f52d5deefd6fa9500cb35a42000834920230b45db55824f5e6562b41ab4a309e"} Jul 20 15:27:42.844130 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.843765 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:42.848214 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.848193 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" Jul 20 15:27:42.858945 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:42.858896 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-grmsw" podStartSLOduration=2.134770755 podStartE2EDuration="3.858881042s" totalImagesPullingTime="1.724110287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:42.858704666 +0000 UTC m=+54.803298033" watchObservedRunningTime="2026-07-20 15:27:42.858881042 +0000 UTC m=+54.803474390" Jul 20 15:27:43.468695 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.468672 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ntqhn_c42f52c3-250c-484e-a5bc-0b3c1c92f4dd/node-ca/0.log" Jul 20 15:27:43.868704 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.868626 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh"] Jul 20 15:27:43.881859 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.881833 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh"] Jul 20 15:27:43.881988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.881926 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:43.883996 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.883951 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 15:27:43.884131 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.884050 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 15:27:43.884182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.884159 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vlxmg\"" Jul 20 15:27:43.884445 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.884430 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 20 15:27:43.884785 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.884751 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 20 15:27:43.884908 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.884821 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 15:27:43.961080 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.961059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:43.961196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.961098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:43.961196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.961123 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cca73d00-0744-4541-80ae-e58215610fc3-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:43.961196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:43.961143 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt486\" (UniqueName: \"kubernetes.io/projected/cca73d00-0744-4541-80ae-e58215610fc3-kube-api-access-rt486\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.062295 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.062269 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.062405 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.062302 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cca73d00-0744-4541-80ae-e58215610fc3-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.062405 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.062326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt486\" (UniqueName: \"kubernetes.io/projected/cca73d00-0744-4541-80ae-e58215610fc3-kube-api-access-rt486\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.062508 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.062399 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.062508 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:44.062403 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 20 15:27:44.062508 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:44.062500 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-tls podName:cca73d00-0744-4541-80ae-e58215610fc3 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:44.562481989 +0000 UTC m=+56.507075339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-hp9dh" (UID: "cca73d00-0744-4541-80ae-e58215610fc3") : secret "prometheus-operator-tls" not found Jul 20 15:27:44.062969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.062952 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cca73d00-0744-4541-80ae-e58215610fc3-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.065689 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.065672 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.070799 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.070759 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt486\" (UniqueName: \"kubernetes.io/projected/cca73d00-0744-4541-80ae-e58215610fc3-kube-api-access-rt486\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.133438 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.133417 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-799b98f4b8-bvbc5"] Jul 20 15:27:44.154891 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.154866 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-799b98f4b8-bvbc5"] Jul 20 15:27:44.154968 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.154946 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.157296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157261 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 15:27:44.157296 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157280 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 15:27:44.157442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157269 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 15:27:44.157442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 15:27:44.157442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157268 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 15:27:44.157783 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157752 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 15:27:44.157839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157761 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v6hm2\"" Jul 20 15:27:44.157883 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.157856 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 15:27:44.162853 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.162826 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-trusted-ca-bundle\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.162956 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.162889 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-service-ca\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.163017 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.162975 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-config\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.163093 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.163067 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-serving-cert\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.163154 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.163110 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-oauth-serving-cert\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.163202 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.163150 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-oauth-config\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.163202 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.163186 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hczv\" (UniqueName: \"kubernetes.io/projected/614abce0-6750-4ac3-a1e3-59c4b0ba7055-kube-api-access-9hczv\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.163494 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.163482 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 15:27:44.263991 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.263966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-serving-cert\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264078 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.263997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-oauth-serving-cert\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264078 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264014 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-oauth-config\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264078 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hczv\" (UniqueName: \"kubernetes.io/projected/614abce0-6750-4ac3-a1e3-59c4b0ba7055-kube-api-access-9hczv\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264078 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264052 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:44.264267 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264098 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-trusted-ca-bundle\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264267 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:44.264246 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 15:27:44.264359 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-service-ca\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264359 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:44.264311 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:48.264291682 +0000 UTC m=+60.208885036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : secret "insights-runtime-extractor-tls" not found Jul 20 15:27:44.264466 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-config\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.264466 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264424 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:44.264563 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:44.264505 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 15:27:44.264563 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:44.264553 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:48.264536362 +0000 UTC m=+60.209129710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : configmap "kube-rbac-proxy" not found Jul 20 15:27:44.264889 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264858 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-oauth-serving-cert\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.265001 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.264981 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-service-ca\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.265136 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.265120 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-config\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.265476 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.265460 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-trusted-ca-bundle\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.267043 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.267022 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-oauth-config\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.267125 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.267084 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-serving-cert\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.269878 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.269858 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lhfxq_5bfe5645-2124-423a-921e-f294097db8d9/serve-healthcheck-canary/0.log" Jul 20 15:27:44.272196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.272177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hczv\" (UniqueName: \"kubernetes.io/projected/614abce0-6750-4ac3-a1e3-59c4b0ba7055-kube-api-access-9hczv\") pod \"console-799b98f4b8-bvbc5\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.464007 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.463955 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:44.567586 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.567425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.569999 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.569981 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cca73d00-0744-4541-80ae-e58215610fc3-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-hp9dh\" (UID: \"cca73d00-0744-4541-80ae-e58215610fc3\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.599098 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.599072 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-799b98f4b8-bvbc5"] Jul 20 15:27:44.602502 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:44.602478 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614abce0_6750_4ac3_a1e3_59c4b0ba7055.slice/crio-2af02efd1af741c276d79cc8a9caac1650185c464afcc4a192a2a981e5295e38 WatchSource:0}: Error finding container 2af02efd1af741c276d79cc8a9caac1650185c464afcc4a192a2a981e5295e38: Status 404 returned error can't find the container with id 2af02efd1af741c276d79cc8a9caac1650185c464afcc4a192a2a981e5295e38 Jul 20 15:27:44.790437 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.790368 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" Jul 20 15:27:44.850507 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.850483 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799b98f4b8-bvbc5" event={"ID":"614abce0-6750-4ac3-a1e3-59c4b0ba7055","Type":"ContainerStarted","Data":"2af02efd1af741c276d79cc8a9caac1650185c464afcc4a192a2a981e5295e38"} Jul 20 15:27:44.932930 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:44.932882 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh"] Jul 20 15:27:44.936765 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:44.936738 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca73d00_0744_4541_80ae_e58215610fc3.slice/crio-71b7909e142223ab971bc261d7f37e91e51b1a27b6a5806ae053b8f8cb21770b WatchSource:0}: Error finding container 71b7909e142223ab971bc261d7f37e91e51b1a27b6a5806ae053b8f8cb21770b: Status 404 returned error can't find the container with id 71b7909e142223ab971bc261d7f37e91e51b1a27b6a5806ae053b8f8cb21770b Jul 20 15:27:45.854855 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:45.854816 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" event={"ID":"cca73d00-0744-4541-80ae-e58215610fc3","Type":"ContainerStarted","Data":"71b7909e142223ab971bc261d7f37e91e51b1a27b6a5806ae053b8f8cb21770b"} Jul 20 15:27:46.864527 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:46.864478 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" event={"ID":"cca73d00-0744-4541-80ae-e58215610fc3","Type":"ContainerStarted","Data":"7a92ab5499ea3867e168d9bef252d98c36d28435a3e20fea1a9bed598b60043f"} Jul 20 15:27:46.864982 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:46.864582 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" event={"ID":"cca73d00-0744-4541-80ae-e58215610fc3","Type":"ContainerStarted","Data":"929c6893809269701a3380c3937f10e2ce20a30ae39d7cbceff89b10ec281228"} Jul 20 15:27:46.879956 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:46.879910 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-hp9dh" podStartSLOduration=2.632733243 podStartE2EDuration="3.879896307s" totalImagesPullingTime="1.247163064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:46.87939417 +0000 UTC m=+58.823987540" watchObservedRunningTime="2026-07-20 15:27:46.879896307 +0000 UTC m=+58.824489796" Jul 20 15:27:47.869003 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:47.868921 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799b98f4b8-bvbc5" event={"ID":"614abce0-6750-4ac3-a1e3-59c4b0ba7055","Type":"ContainerStarted","Data":"88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98"} Jul 20 15:27:47.889191 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:47.889139 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-799b98f4b8-bvbc5" podStartSLOduration=1.043044107 podStartE2EDuration="3.889120206s" totalImagesPullingTime="2.846076099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:47.888239921 +0000 UTC m=+59.832833292" watchObservedRunningTime="2026-07-20 15:27:47.889120206 +0000 UTC m=+59.833713575" Jul 20 15:27:48.300493 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:48.300462 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:48.300642 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:48.300502 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:48.300642 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:48.300576 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 15:27:48.300642 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:48.300607 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 15:27:48.300741 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:48.300645 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:56.300628954 +0000 UTC m=+68.245222302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : configmap "kube-rbac-proxy" not found Jul 20 15:27:48.300741 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:48.300659 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls podName:493dd2d2-5ce0-417c-8598-444afa797b51 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:56.30065269 +0000 UTC m=+68.245246037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nghgd" (UID: "493dd2d2-5ce0-417c-8598-444afa797b51") : secret "insights-runtime-extractor-tls" not found Jul 20 15:27:49.341022 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.340991 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l"] Jul 20 15:27:49.343618 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.343603 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.346035 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.346018 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-l6zj5\"" Jul 20 15:27:49.346259 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.346245 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 15:27:49.346308 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.346247 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 15:27:49.357258 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.357237 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l"] Jul 20 15:27:49.388702 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.388681 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-srr59"] Jul 20 15:27:49.390891 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.390876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.392987 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.392904 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 15:27:49.392987 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.392972 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4jxgh\"" Jul 20 15:27:49.393134 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.392912 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 15:27:49.393263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.393240 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 15:27:49.410180 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-wtmp\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410283 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410283 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410221 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-tls\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410283 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sx7n\" (UniqueName: \"kubernetes.io/projected/9e05225f-7edc-4014-a59c-a094a6de013d-kube-api-access-2sx7n\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.410283 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410276 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zt2l\" (UniqueName: \"kubernetes.io/projected/bc87ef23-5059-45dc-8451-f31e41dd50f0-kube-api-access-5zt2l\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410456 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e05225f-7edc-4014-a59c-a094a6de013d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.410456 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410321 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410456 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410388 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e05225f-7edc-4014-a59c-a094a6de013d-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.410456 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410436 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc87ef23-5059-45dc-8451-f31e41dd50f0-metrics-client-ca\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410574 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410477 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e05225f-7edc-4014-a59c-a094a6de013d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.410574 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410505 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-textfile\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410574 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410530 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-root\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.410574 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.410554 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-sys\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.511378 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.511541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-tls\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.511541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511411 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sx7n\" (UniqueName: \"kubernetes.io/projected/9e05225f-7edc-4014-a59c-a094a6de013d-kube-api-access-2sx7n\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.511541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511430 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zt2l\" (UniqueName: \"kubernetes.io/projected/bc87ef23-5059-45dc-8451-f31e41dd50f0-kube-api-access-5zt2l\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.511541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511469 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e05225f-7edc-4014-a59c-a094a6de013d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.511541 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:49.511536 2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 15:27:49.511831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.511831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e05225f-7edc-4014-a59c-a094a6de013d-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.511831 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:27:49.511696 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-tls podName:bc87ef23-5059-45dc-8451-f31e41dd50f0 nodeName:}" failed. No retries permitted until 2026-07-20 15:27:50.0116767 +0000 UTC m=+61.956270053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-tls") pod "node-exporter-srr59" (UID: "bc87ef23-5059-45dc-8451-f31e41dd50f0") : secret "node-exporter-tls" not found Jul 20 15:27:49.511831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511739 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc87ef23-5059-45dc-8451-f31e41dd50f0-metrics-client-ca\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.511831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511824 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e05225f-7edc-4014-a59c-a094a6de013d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.512085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-textfile\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-root\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511940 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-sys\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.511990 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-wtmp\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512277 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512106 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-wtmp\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512277 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512199 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-root\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512376 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512271 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512376 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512342 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc87ef23-5059-45dc-8451-f31e41dd50f0-metrics-client-ca\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512376 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512346 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc87ef23-5059-45dc-8451-f31e41dd50f0-sys\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.512376 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512362 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e05225f-7edc-4014-a59c-a094a6de013d-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.512517 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.512475 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-textfile\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.514420 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.514403 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.514934 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.514917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e05225f-7edc-4014-a59c-a094a6de013d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.515046 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.515029 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e05225f-7edc-4014-a59c-a094a6de013d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.526370 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.526344 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zt2l\" (UniqueName: \"kubernetes.io/projected/bc87ef23-5059-45dc-8451-f31e41dd50f0-kube-api-access-5zt2l\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:49.526468 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.526452 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sx7n\" (UniqueName: \"kubernetes.io/projected/9e05225f-7edc-4014-a59c-a094a6de013d-kube-api-access-2sx7n\") pod \"openshift-state-metrics-79f6958cdf-zcw8l\" (UID: \"9e05225f-7edc-4014-a59c-a094a6de013d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.652503 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.652480 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" Jul 20 15:27:49.786041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.786013 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l"] Jul 20 15:27:49.789298 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:49.789269 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e05225f_7edc_4014_a59c_a094a6de013d.slice/crio-7a7d92bbd7cd6c13c2bfb5b582977f493544a4565ddfb3a612d5fa4df85a53d5 WatchSource:0}: Error finding container 7a7d92bbd7cd6c13c2bfb5b582977f493544a4565ddfb3a612d5fa4df85a53d5: Status 404 returned error can't find the container with id 7a7d92bbd7cd6c13c2bfb5b582977f493544a4565ddfb3a612d5fa4df85a53d5 Jul 20 15:27:49.877220 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.877194 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" event={"ID":"9e05225f-7edc-4014-a59c-a094a6de013d","Type":"ContainerStarted","Data":"c4934718a6983504b80c087ebc08a58fa37069a768e7138f6519e05f1ff93d02"} Jul 20 15:27:49.877302 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:49.877247 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" event={"ID":"9e05225f-7edc-4014-a59c-a094a6de013d","Type":"ContainerStarted","Data":"7a7d92bbd7cd6c13c2bfb5b582977f493544a4565ddfb3a612d5fa4df85a53d5"} Jul 20 15:27:50.016271 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.016178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-tls\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:50.018532 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.018503 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc87ef23-5059-45dc-8451-f31e41dd50f0-node-exporter-tls\") pod \"node-exporter-srr59\" (UID: \"bc87ef23-5059-45dc-8451-f31e41dd50f0\") " pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:50.299897 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.299824 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srr59" Jul 20 15:27:50.308876 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:50.308850 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc87ef23_5059_45dc_8451_f31e41dd50f0.slice/crio-52e40870734a2545484dc4e06129b3b148ad0a537956376b22f600c0d9864ffd WatchSource:0}: Error finding container 52e40870734a2545484dc4e06129b3b148ad0a537956376b22f600c0d9864ffd: Status 404 returned error can't find the container with id 52e40870734a2545484dc4e06129b3b148ad0a537956376b22f600c0d9864ffd Jul 20 15:27:50.478983 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.478281 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:27:50.486415 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.486389 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.490398 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.490250 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 15:27:50.490398 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.490276 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 15:27:50.490398 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.490346 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 15:27:50.490621 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.490595 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wstnt\"" Jul 20 15:27:50.490852 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.490830 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 15:27:50.491004 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.490912 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 15:27:50.491099 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.491082 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 15:27:50.491315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.491292 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 15:27:50.491402 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.491358 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 15:27:50.491402 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.491295 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 15:27:50.508417 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.508392 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:27:50.521271 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521242 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521368 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521284 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-config-volume\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521368 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521315 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/979890d9-4a3c-4257-be1e-4a26f6b91033-config-out\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521368 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521333 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521372 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-web-config\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521397 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/979890d9-4a3c-4257-be1e-4a26f6b91033-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521435 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vkc4\" (UniqueName: \"kubernetes.io/projected/979890d9-4a3c-4257-be1e-4a26f6b91033-kube-api-access-5vkc4\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521476 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521502 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/979890d9-4a3c-4257-be1e-4a26f6b91033-tls-assets\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521554 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/979890d9-4a3c-4257-be1e-4a26f6b91033-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.521707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.521568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/979890d9-4a3c-4257-be1e-4a26f6b91033-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623002 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.622970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623157 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-web-config\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623335 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623193 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/979890d9-4a3c-4257-be1e-4a26f6b91033-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623335 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vkc4\" (UniqueName: \"kubernetes.io/projected/979890d9-4a3c-4257-be1e-4a26f6b91033-kube-api-access-5vkc4\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/979890d9-4a3c-4257-be1e-4a26f6b91033-tls-assets\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623671 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/979890d9-4a3c-4257-be1e-4a26f6b91033-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/979890d9-4a3c-4257-be1e-4a26f6b91033-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.623962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.623754 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.625419 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.624390 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/979890d9-4a3c-4257-be1e-4a26f6b91033-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.625419 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.624401 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/979890d9-4a3c-4257-be1e-4a26f6b91033-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.625419 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.624821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-config-volume\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.625419 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.624872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/979890d9-4a3c-4257-be1e-4a26f6b91033-config-out\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.625419 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.625375 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/979890d9-4a3c-4257-be1e-4a26f6b91033-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.626487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.626441 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-web-config\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.626942 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.626899 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.627230 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.627209 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.627320 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.627297 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.627563 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.627540 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/979890d9-4a3c-4257-be1e-4a26f6b91033-tls-assets\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.627791 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.627750 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.627912 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.627887 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-config-volume\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.628396 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.628374 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/979890d9-4a3c-4257-be1e-4a26f6b91033-config-out\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.628916 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.628895 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/979890d9-4a3c-4257-be1e-4a26f6b91033-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.632085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.632063 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vkc4\" (UniqueName: \"kubernetes.io/projected/979890d9-4a3c-4257-be1e-4a26f6b91033-kube-api-access-5vkc4\") pod \"alertmanager-main-0\" (UID: \"979890d9-4a3c-4257-be1e-4a26f6b91033\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.785694 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.785667 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hzlp" Jul 20 15:27:50.797864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.797822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 15:27:50.842361 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.842316 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2c5dx" Jul 20 15:27:50.884485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.884400 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" event={"ID":"9e05225f-7edc-4014-a59c-a094a6de013d","Type":"ContainerStarted","Data":"1d723064af973b80f8e9d7df7a66c4ef6d6c7713e9076adf2af959645c70eb49"} Jul 20 15:27:50.887599 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:50.885877 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srr59" event={"ID":"bc87ef23-5059-45dc-8451-f31e41dd50f0","Type":"ContainerStarted","Data":"52e40870734a2545484dc4e06129b3b148ad0a537956376b22f600c0d9864ffd"} Jul 20 15:27:51.140911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:51.140817 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 15:27:51.144577 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:51.144543 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979890d9_4a3c_4257_be1e_4a26f6b91033.slice/crio-6b578db860c7f66b749866917f4c08a57b0b311f50eb4c436a56b8878aea9b2c WatchSource:0}: Error finding container 6b578db860c7f66b749866917f4c08a57b0b311f50eb4c436a56b8878aea9b2c: Status 404 returned error can't find the container with id 6b578db860c7f66b749866917f4c08a57b0b311f50eb4c436a56b8878aea9b2c Jul 20 15:27:51.891062 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:51.890975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"6b578db860c7f66b749866917f4c08a57b0b311f50eb4c436a56b8878aea9b2c"} Jul 20 15:27:51.892953 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:51.892918 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" event={"ID":"9e05225f-7edc-4014-a59c-a094a6de013d","Type":"ContainerStarted","Data":"cfd3590b3ad8ba4e2b9cc102b793ede67b8b0e3ab26548e4e178e6e458b24f50"} Jul 20 15:27:51.918521 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:51.918480 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zcw8l" podStartSLOduration=1.178075074 podStartE2EDuration="2.91846376s" totalImagesPullingTime="1.740388686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:51.917810641 +0000 UTC m=+63.862404017" watchObservedRunningTime="2026-07-20 15:27:51.91846376 +0000 UTC m=+63.863057130" Jul 20 15:27:52.898185 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:52.898151 2557 generic.go:356] "Generic (PLEG): container finished" podID="979890d9-4a3c-4257-be1e-4a26f6b91033" containerID="caf4d54614c19040328ae254503f371d5682cf7d57d411145fb7037ebf4d5203" exitCode=0 Jul 20 15:27:52.898604 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:52.898237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerDied","Data":"caf4d54614c19040328ae254503f371d5682cf7d57d411145fb7037ebf4d5203"} Jul 20 15:27:52.899989 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:52.899966 2557 generic.go:356] "Generic (PLEG): container finished" podID="bc87ef23-5059-45dc-8451-f31e41dd50f0" containerID="f4be5824ce1b674765883444f60a7c69ec6b597ae664c49e6186f06742edd645" exitCode=0 Jul 20 15:27:52.900103 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:52.900042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srr59" event={"ID":"bc87ef23-5059-45dc-8451-f31e41dd50f0","Type":"ContainerDied","Data":"f4be5824ce1b674765883444f60a7c69ec6b597ae664c49e6186f06742edd645"} Jul 20 15:27:53.252011 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.251938 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:53.252011 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.251997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:53.253969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.253950 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 15:27:53.254028 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.253992 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 15:27:53.265321 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.265302 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515a8194-00d7-4d90-8091-44183e8d3bb4-metrics-certs\") pod \"network-metrics-daemon-gbvtz\" (UID: \"515a8194-00d7-4d90-8091-44183e8d3bb4\") " pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:53.265425 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.265380 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/598ac123-40fd-474c-b70e-8877d0aef642-original-pull-secret\") pod \"global-pull-secret-syncer-ddwq7\" (UID: \"598ac123-40fd-474c-b70e-8877d0aef642\") " pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:53.352871 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.352829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:53.354858 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.354842 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 15:27:53.366039 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.366016 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 15:27:53.377198 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.377120 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c27t\" (UniqueName: \"kubernetes.io/projected/65630c99-f328-4ee7-96c6-7b4b282c1552-kube-api-access-5c27t\") pod \"network-check-target-2bk9l\" (UID: \"65630c99-f328-4ee7-96c6-7b4b282c1552\") " pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:53.476147 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.476112 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tnhrs\"" Jul 20 15:27:53.481869 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.481844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddwq7" Jul 20 15:27:53.484723 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.484705 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:53.490524 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.490505 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zgzvz\"" Jul 20 15:27:53.499529 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.499508 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gbvtz" Jul 20 15:27:53.636885 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.636801 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2bk9l"] Jul 20 15:27:53.640990 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:53.640962 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65630c99_f328_4ee7_96c6_7b4b282c1552.slice/crio-7e00e2d03d26747a797fc648df2d67a02ab8cd75d226f5e6e2bdbea95afc8085 WatchSource:0}: Error finding container 7e00e2d03d26747a797fc648df2d67a02ab8cd75d226f5e6e2bdbea95afc8085: Status 404 returned error can't find the container with id 7e00e2d03d26747a797fc648df2d67a02ab8cd75d226f5e6e2bdbea95afc8085 Jul 20 15:27:53.648262 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.648234 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddwq7"] Jul 20 15:27:53.652029 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:53.652003 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598ac123_40fd_474c_b70e_8877d0aef642.slice/crio-6c2e0c76c7f7a207e76c3850b295b18a932be58e60b636ad806a477c896c8b15 WatchSource:0}: Error finding container 6c2e0c76c7f7a207e76c3850b295b18a932be58e60b636ad806a477c896c8b15: Status 404 returned error can't find the container with id 6c2e0c76c7f7a207e76c3850b295b18a932be58e60b636ad806a477c896c8b15 Jul 20 15:27:53.668397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.668374 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gbvtz"] Jul 20 15:27:53.671226 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:53.671150 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515a8194_00d7_4d90_8091_44183e8d3bb4.slice/crio-ff305891707a0f1392a1febde4d832b421c299c744c007885b6adeed7ef9605b WatchSource:0}: Error finding container ff305891707a0f1392a1febde4d832b421c299c744c007885b6adeed7ef9605b: Status 404 returned error can't find the container with id ff305891707a0f1392a1febde4d832b421c299c744c007885b6adeed7ef9605b Jul 20 15:27:53.766610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.766551 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b5cd9d9f4-bgv97"] Jul 20 15:27:53.769441 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.769423 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.771972 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.771952 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-29xtl\"" Jul 20 15:27:53.772067 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.772040 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 20 15:27:53.772286 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.772267 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 15:27:53.772400 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.772360 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 20 15:27:53.772514 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.772364 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bmqgskda7lqqp\"" Jul 20 15:27:53.772514 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.772474 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 20 15:27:53.778977 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.778958 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b5cd9d9f4-bgv97"] Jul 20 15:27:53.859987 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.859961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-secret-metrics-server-client-certs\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.860094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.859991 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1c314e-ac25-4670-b116-1c703d9181cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.860094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.860014 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-client-ca-bundle\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.860094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.860051 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9d1c314e-ac25-4670-b116-1c703d9181cd-metrics-server-audit-profiles\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.860094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.860068 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn4qp\" (UniqueName: \"kubernetes.io/projected/9d1c314e-ac25-4670-b116-1c703d9181cd-kube-api-access-kn4qp\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.860233 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.860136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-secret-metrics-server-tls\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.860233 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.860153 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9d1c314e-ac25-4670-b116-1c703d9181cd-audit-log\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.904216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.904189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gbvtz" event={"ID":"515a8194-00d7-4d90-8091-44183e8d3bb4","Type":"ContainerStarted","Data":"ff305891707a0f1392a1febde4d832b421c299c744c007885b6adeed7ef9605b"} Jul 20 15:27:53.906470 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.906451 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srr59" event={"ID":"bc87ef23-5059-45dc-8451-f31e41dd50f0","Type":"ContainerStarted","Data":"7493cbc078f315d351db6353e2deff823af489de5cb5a36806fa094fcc05e698"} Jul 20 15:27:53.906524 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.906489 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srr59" event={"ID":"bc87ef23-5059-45dc-8451-f31e41dd50f0","Type":"ContainerStarted","Data":"ac24b67e4d9e1e0494e9c7def75da68fc70e050dcf703e191e03187a05b6c739"} Jul 20 15:27:53.907613 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.907593 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddwq7" event={"ID":"598ac123-40fd-474c-b70e-8877d0aef642","Type":"ContainerStarted","Data":"6c2e0c76c7f7a207e76c3850b295b18a932be58e60b636ad806a477c896c8b15"} Jul 20 15:27:53.908689 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.908665 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bk9l" event={"ID":"65630c99-f328-4ee7-96c6-7b4b282c1552","Type":"ContainerStarted","Data":"7e00e2d03d26747a797fc648df2d67a02ab8cd75d226f5e6e2bdbea95afc8085"} Jul 20 15:27:53.926357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.926302 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-srr59" podStartSLOduration=3.205427429 podStartE2EDuration="4.926285228s" totalImagesPullingTime="1.640504196s" totalInitContainerRuntime="80.353603ms" isStatefulPod=true podCreationTimestamp="2026-07-20 15:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:53.924264104 +0000 UTC m=+65.868857474" watchObservedRunningTime="2026-07-20 15:27:53.926285228 +0000 UTC m=+65.870878598" Jul 20 15:27:53.961192 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-secret-metrics-server-client-certs\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.961308 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961198 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1c314e-ac25-4670-b116-1c703d9181cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.961308 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-client-ca-bundle\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.961308 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961236 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9d1c314e-ac25-4670-b116-1c703d9181cd-metrics-server-audit-profiles\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.961308 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961252 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn4qp\" (UniqueName: \"kubernetes.io/projected/9d1c314e-ac25-4670-b116-1c703d9181cd-kube-api-access-kn4qp\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.961789 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-secret-metrics-server-tls\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.961789 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.961701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9d1c314e-ac25-4670-b116-1c703d9181cd-audit-log\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.962058 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.962022 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1c314e-ac25-4670-b116-1c703d9181cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.962207 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.962175 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9d1c314e-ac25-4670-b116-1c703d9181cd-audit-log\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.962428 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.962409 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9d1c314e-ac25-4670-b116-1c703d9181cd-metrics-server-audit-profiles\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.964332 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.964308 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-secret-metrics-server-tls\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.964529 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.964511 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-secret-metrics-server-client-certs\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.964608 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.964552 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d1c314e-ac25-4670-b116-1c703d9181cd-client-ca-bundle\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:53.971947 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:53.971924 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn4qp\" (UniqueName: \"kubernetes.io/projected/9d1c314e-ac25-4670-b116-1c703d9181cd-kube-api-access-kn4qp\") pod \"metrics-server-b5cd9d9f4-bgv97\" (UID: \"9d1c314e-ac25-4670-b116-1c703d9181cd\") " pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:54.081194 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.081119 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:27:54.236555 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.236521 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b5cd9d9f4-bgv97"] Jul 20 15:27:54.464987 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.464952 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:54.466353 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.466246 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:54.472110 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.472083 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:54.609344 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:54.609310 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1c314e_ac25_4670_b116_1c703d9181cd.slice/crio-ad39a7d30507d06a49f87d87a9d93deb6e3a36ad63ebfc937651c9c2d92802f7 WatchSource:0}: Error finding container ad39a7d30507d06a49f87d87a9d93deb6e3a36ad63ebfc937651c9c2d92802f7: Status 404 returned error can't find the container with id ad39a7d30507d06a49f87d87a9d93deb6e3a36ad63ebfc937651c9c2d92802f7 Jul 20 15:27:54.915202 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.915172 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" event={"ID":"9d1c314e-ac25-4670-b116-1c703d9181cd","Type":"ContainerStarted","Data":"ad39a7d30507d06a49f87d87a9d93deb6e3a36ad63ebfc937651c9c2d92802f7"} Jul 20 15:27:54.920841 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:54.920820 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:27:55.936485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.936210 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"6729952807eca8dcc334673d7c3412d568d39043ce3f64a1ddc737ea301431fc"} Jul 20 15:27:55.936485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.936297 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"b8debe15516aecba6e77cafd8da7cbdcf111cd11f4512fdbeb66abe59a7476f0"} Jul 20 15:27:55.936485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.936337 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"81592733a7339f6652781d47da3f82251d59d7459e9d449ab121ea19bc2d5c14"} Jul 20 15:27:55.936485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.936376 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"5934890970f52e3ced9fdb28d019aa7a0239a44da4b7873390e5c161706c839c"} Jul 20 15:27:55.936485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.936414 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"b735813784d4ebd219f4d4d924b41664adeb77c53eddf79afbd08acdc6c90f01"} Jul 20 15:27:55.944812 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.944632 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gbvtz" event={"ID":"515a8194-00d7-4d90-8091-44183e8d3bb4","Type":"ContainerStarted","Data":"9069bae7f4d6121649f62f0e35a91d73ff0e74a4da5b55018f5620781664ba02"} Jul 20 15:27:55.944812 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:55.944687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gbvtz" event={"ID":"515a8194-00d7-4d90-8091-44183e8d3bb4","Type":"ContainerStarted","Data":"28076ad12169e9bdd2bdaa02c297f6f15244d0b340bedfce41594fca2a47d8f4"} Jul 20 15:27:56.384725 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:56.384692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:56.384910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:56.384792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:56.385510 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:56.385482 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/493dd2d2-5ce0-417c-8598-444afa797b51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:56.396928 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:56.396894 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/493dd2d2-5ce0-417c-8598-444afa797b51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nghgd\" (UID: \"493dd2d2-5ce0-417c-8598-444afa797b51\") " pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:56.410243 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:56.410217 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9rlnk\"" Jul 20 15:27:56.418839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:56.418817 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nghgd" Jul 20 15:27:59.380333 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.380278 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gbvtz" podStartSLOduration=69.16055037 podStartE2EDuration="1m10.380255116s" totalImagesPullingTime="1.219704746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:55.961682951 +0000 UTC m=+67.906276322" watchObservedRunningTime="2026-07-20 15:27:59.380255116 +0000 UTC m=+71.324848487" Jul 20 15:27:59.382075 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.380943 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57d75d484c-jtqw2"] Jul 20 15:27:59.383993 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.383927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.397066 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.397026 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d75d484c-jtqw2"] Jul 20 15:27:59.451267 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.451121 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nghgd"] Jul 20 15:27:59.508099 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-config\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.508215 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-oauth-config\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.508215 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508168 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-serving-cert\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.508215 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508192 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-trusted-ca-bundle\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.508371 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508221 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-oauth-serving-cert\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.508371 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4wxj\" (UniqueName: \"kubernetes.io/projected/6e9dc728-8b40-40c9-b778-d9005aef3c0f-kube-api-access-v4wxj\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.508371 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.508304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-service-ca\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609067 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.608970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-config\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609067 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609018 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-oauth-config\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609067 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609054 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-serving-cert\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609076 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-trusted-ca-bundle\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609101 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-oauth-serving-cert\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4wxj\" (UniqueName: \"kubernetes.io/projected/6e9dc728-8b40-40c9-b778-d9005aef3c0f-kube-api-access-v4wxj\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609170 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-service-ca\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.609943 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609914 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-oauth-serving-cert\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.610064 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.609950 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-service-ca\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.610207 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.610164 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-trusted-ca-bundle\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.610264 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.610221 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-config\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.611754 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.611722 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-serving-cert\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.612252 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.612233 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-oauth-config\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.617279 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.617255 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4wxj\" (UniqueName: \"kubernetes.io/projected/6e9dc728-8b40-40c9-b778-d9005aef3c0f-kube-api-access-v4wxj\") pod \"console-57d75d484c-jtqw2\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.722094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.722059 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:27:59.847839 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.847799 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d75d484c-jtqw2"] Jul 20 15:27:59.850627 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:27:59.850596 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9dc728_8b40_40c9_b778_d9005aef3c0f.slice/crio-0cea90a553fce26c2652b051b0a4bf2ce5984c9ecb860093a131f4d147c94c82 WatchSource:0}: Error finding container 0cea90a553fce26c2652b051b0a4bf2ce5984c9ecb860093a131f4d147c94c82: Status 404 returned error can't find the container with id 0cea90a553fce26c2652b051b0a4bf2ce5984c9ecb860093a131f4d147c94c82 Jul 20 15:27:59.971371 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.970590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddwq7" event={"ID":"598ac123-40fd-474c-b70e-8877d0aef642","Type":"ContainerStarted","Data":"4f1e5f3e16517f89d41e5e9b8fb64e08f27a1996d2fd1d601ed89dcf9ffeb3b1"} Jul 20 15:27:59.975356 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.975215 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75d484c-jtqw2" event={"ID":"6e9dc728-8b40-40c9-b778-d9005aef3c0f","Type":"ContainerStarted","Data":"c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76"} Jul 20 15:27:59.975356 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.975265 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75d484c-jtqw2" event={"ID":"6e9dc728-8b40-40c9-b778-d9005aef3c0f","Type":"ContainerStarted","Data":"0cea90a553fce26c2652b051b0a4bf2ce5984c9ecb860093a131f4d147c94c82"} Jul 20 15:27:59.978824 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.978494 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2bk9l" event={"ID":"65630c99-f328-4ee7-96c6-7b4b282c1552","Type":"ContainerStarted","Data":"df0d2c4253f1be879ebcfd809225b325a68957dd50800730a601232c69d50e12"} Jul 20 15:27:59.978824 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.978763 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:27:59.984129 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.983653 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" event={"ID":"9d1c314e-ac25-4670-b116-1c703d9181cd","Type":"ContainerStarted","Data":"6ed452ecbfa4c5ee25b399f80e8c9c66e36dc188a743585891982e6d75e7def3"} Jul 20 15:27:59.985550 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.985497 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nghgd" event={"ID":"493dd2d2-5ce0-417c-8598-444afa797b51","Type":"ContainerStarted","Data":"ebb3646ccfabf041686bd6742b6ec6f6c2aa8569adec9f07c5f3421922a50ca0"} Jul 20 15:27:59.989328 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.989292 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ddwq7" podStartSLOduration=65.259005165 podStartE2EDuration="1m10.989277956s" totalImagesPullingTime="5.730272791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:27:59.987228192 +0000 UTC m=+71.931821565" watchObservedRunningTime="2026-07-20 15:27:59.989277956 +0000 UTC m=+71.933871366" Jul 20 15:27:59.993178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:27:59.992954 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"979890d9-4a3c-4257-be1e-4a26f6b91033","Type":"ContainerStarted","Data":"39faa592cf4626ba06ab1a7d92e78698369383ffe0058b4fd87edab2d9a49d75"} Jul 20 15:28:00.004630 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:00.004523 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2bk9l" podStartSLOduration=65.333159463 podStartE2EDuration="1m11.004505856s" totalImagesPullingTime="5.671346393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:00.004191438 +0000 UTC m=+71.948784809" watchObservedRunningTime="2026-07-20 15:28:00.004505856 +0000 UTC m=+71.949099229" Jul 20 15:28:00.021815 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:00.021652 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" podStartSLOduration=2.328350387 podStartE2EDuration="7.021642712s" totalImagesPullingTime="4.693292325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:00.021292899 +0000 UTC m=+71.965886267" watchObservedRunningTime="2026-07-20 15:28:00.021642712 +0000 UTC m=+71.966236075" Jul 20 15:28:00.042816 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:00.042735 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57d75d484c-jtqw2" podStartSLOduration=1.042719521 podStartE2EDuration="1.042719521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:00.04163772 +0000 UTC m=+71.986231092" watchObservedRunningTime="2026-07-20 15:28:00.042719521 +0000 UTC m=+71.987312892" Jul 20 15:28:00.068016 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:00.067969 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.465036628 podStartE2EDuration="10.067949295s" totalImagesPullingTime="7.586512552s" totalInitContainerRuntime="16.400115ms" isStatefulPod=false podCreationTimestamp="2026-07-20 15:27:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:00.066243406 +0000 UTC m=+72.010836793" watchObservedRunningTime="2026-07-20 15:28:00.067949295 +0000 UTC m=+72.012542666" Jul 20 15:28:05.018974 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:05.018935 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nghgd" event={"ID":"493dd2d2-5ce0-417c-8598-444afa797b51","Type":"ContainerStarted","Data":"44d06e0fef561e34be286e65ab57d4e7baa0def0c910bbcde3f72f034cc9fa0a"} Jul 20 15:28:08.031741 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:08.031702 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nghgd" event={"ID":"493dd2d2-5ce0-417c-8598-444afa797b51","Type":"ContainerStarted","Data":"e10d2a95d19c08d25e0b4b17f093e8a8e2a5640b22b3e49617aa2de12f728e95"} Jul 20 15:28:09.722598 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:09.722469 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:28:09.722598 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:09.722546 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:28:09.727541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:09.727520 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:28:10.044811 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:10.044711 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:28:10.110888 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:10.110855 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-799b98f4b8-bvbc5"] Jul 20 15:28:12.053616 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:12.053574 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nghgd" event={"ID":"493dd2d2-5ce0-417c-8598-444afa797b51","Type":"ContainerStarted","Data":"2e538c925d7e31c889e30f0364678a04606e821a74ffefd20464ac0dc4c5b4c4"} Jul 20 15:28:12.071566 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:12.071518 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nghgd" podStartSLOduration=20.251654355 podStartE2EDuration="32.071501364s" totalImagesPullingTime="11.819847009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 15:27:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:12.069636732 +0000 UTC m=+84.014230102" watchObservedRunningTime="2026-07-20 15:28:12.071501364 +0000 UTC m=+84.016094735" Jul 20 15:28:14.081407 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:14.081373 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:28:14.081889 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:14.081431 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:28:31.001350 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:31.001315 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2bk9l" Jul 20 15:28:34.086666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:34.086636 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:28:34.090512 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:34.090491 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b5cd9d9f4-bgv97" Jul 20 15:28:35.134546 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.134504 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-799b98f4b8-bvbc5" podUID="614abce0-6750-4ac3-a1e3-59c4b0ba7055" containerName="console" containerID="cri-o://88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98" gracePeriod=15 Jul 20 15:28:35.420463 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.420398 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:28:35.599620 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599584 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-config\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.599818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599633 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-oauth-config\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.599818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599658 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-oauth-serving-cert\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.599818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599676 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hczv\" (UniqueName: \"kubernetes.io/projected/614abce0-6750-4ac3-a1e3-59c4b0ba7055-kube-api-access-9hczv\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.599818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599695 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-trusted-ca-bundle\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.599818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599711 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-service-ca\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.599818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.599794 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-serving-cert\") pod \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\" (UID: \"614abce0-6750-4ac3-a1e3-59c4b0ba7055\") " Jul 20 15:28:35.600151 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.600077 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-config" (OuterVolumeSpecName: "console-config") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:28:35.600203 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.600174 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:28:35.600304 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.600278 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-service-ca" (OuterVolumeSpecName: "service-ca") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:28:35.600447 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.600427 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:28:35.602291 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.602257 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:28:35.602433 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.602413 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:28:35.602501 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.602468 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614abce0-6750-4ac3-a1e3-59c4b0ba7055-kube-api-access-9hczv" (OuterVolumeSpecName: "kube-api-access-9hczv") pod "614abce0-6750-4ac3-a1e3-59c4b0ba7055" (UID: "614abce0-6750-4ac3-a1e3-59c4b0ba7055"). InnerVolumeSpecName "kube-api-access-9hczv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:28:35.700918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700845 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:35.700918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700879 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:35.700918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700893 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614abce0-6750-4ac3-a1e3-59c4b0ba7055-console-oauth-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:35.700918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700905 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-oauth-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:35.700918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700916 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hczv\" (UniqueName: \"kubernetes.io/projected/614abce0-6750-4ac3-a1e3-59c4b0ba7055-kube-api-access-9hczv\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:35.701146 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700929 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-trusted-ca-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:35.701146 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:35.700943 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614abce0-6750-4ac3-a1e3-59c4b0ba7055-service-ca\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:28:36.144804 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.144757 2557 generic.go:356] "Generic (PLEG): container finished" podID="614abce0-6750-4ac3-a1e3-59c4b0ba7055" containerID="88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98" exitCode=0 Jul 20 15:28:36.145183 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.144811 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799b98f4b8-bvbc5" Jul 20 15:28:36.145183 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.144815 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799b98f4b8-bvbc5" event={"ID":"614abce0-6750-4ac3-a1e3-59c4b0ba7055","Type":"ContainerDied","Data":"88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98"} Jul 20 15:28:36.145183 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.144913 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799b98f4b8-bvbc5" event={"ID":"614abce0-6750-4ac3-a1e3-59c4b0ba7055","Type":"ContainerDied","Data":"2af02efd1af741c276d79cc8a9caac1650185c464afcc4a192a2a981e5295e38"} Jul 20 15:28:36.145183 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.144957 2557 scope.go:117] "RemoveContainer" containerID="88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98" Jul 20 15:28:36.159673 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.159657 2557 scope.go:117] "RemoveContainer" containerID="88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98" Jul 20 15:28:36.159968 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:28:36.159950 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98\": container with ID starting with 88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98 not found: ID does not exist" containerID="88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98" Jul 20 15:28:36.160014 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.159978 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98"} err="failed to get container status \"88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98\": rpc error: code = NotFound desc = could not find container \"88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98\": container with ID starting with 88ef251b3e9e1efc0f4c8954e09a9fa027b62ae65a0b45411592b2faa85b0a98 not found: ID does not exist" Jul 20 15:28:36.164786 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.164745 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-799b98f4b8-bvbc5"] Jul 20 15:28:36.168179 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.168161 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-799b98f4b8-bvbc5"] Jul 20 15:28:36.564195 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:36.564138 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614abce0-6750-4ac3-a1e3-59c4b0ba7055" path="/var/lib/kubelet/pods/614abce0-6750-4ac3-a1e3-59c4b0ba7055/volumes" Jul 20 15:28:53.578793 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.578737 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f499b659c-7p4tj"] Jul 20 15:28:53.579196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.579007 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614abce0-6750-4ac3-a1e3-59c4b0ba7055" containerName="console" Jul 20 15:28:53.579196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.579017 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="614abce0-6750-4ac3-a1e3-59c4b0ba7055" containerName="console" Jul 20 15:28:53.579196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.579070 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="614abce0-6750-4ac3-a1e3-59c4b0ba7055" containerName="console" Jul 20 15:28:53.608371 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.608331 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f499b659c-7p4tj"] Jul 20 15:28:53.608545 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.608460 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.611224 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.611200 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vmt79\"" Jul 20 15:28:53.611379 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.611238 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 20 15:28:53.611379 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.611345 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 20 15:28:53.611502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.611428 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 20 15:28:53.611554 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.611466 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 20 15:28:53.611732 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.611718 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 20 15:28:53.616581 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.616557 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 20 15:28:53.628229 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.628205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.628430 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.628413 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-federate-client-tls\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.628617 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.628601 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-serving-certs-ca-bundle\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.628719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.628705 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.628905 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.628887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clpqr\" (UniqueName: \"kubernetes.io/projected/343c9bf5-baf3-4b4c-9255-8ddc7c560026-kube-api-access-clpqr\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.629060 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.629043 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-secret-telemeter-client\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.629170 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.629156 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-telemeter-client-tls\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.629262 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.629249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-metrics-client-ca\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730157 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-serving-certs-ca-bundle\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730281 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730163 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730281 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730193 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clpqr\" (UniqueName: \"kubernetes.io/projected/343c9bf5-baf3-4b4c-9255-8ddc7c560026-kube-api-access-clpqr\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730281 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-secret-telemeter-client\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730281 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-telemeter-client-tls\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730350 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-metrics-client-ca\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730408 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730444 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-federate-client-tls\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.730921 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.730897 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-serving-certs-ca-bundle\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.731041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.731021 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-metrics-client-ca\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.731202 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.731186 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/343c9bf5-baf3-4b4c-9255-8ddc7c560026-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.733034 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.733012 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.733128 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.733067 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-federate-client-tls\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.733312 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.733293 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-secret-telemeter-client\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.733354 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.733325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/343c9bf5-baf3-4b4c-9255-8ddc7c560026-telemeter-client-tls\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.738564 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.738544 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clpqr\" (UniqueName: \"kubernetes.io/projected/343c9bf5-baf3-4b4c-9255-8ddc7c560026-kube-api-access-clpqr\") pod \"telemeter-client-7f499b659c-7p4tj\" (UID: \"343c9bf5-baf3-4b4c-9255-8ddc7c560026\") " pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:53.920181 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:53.920160 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" Jul 20 15:28:54.064615 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:54.064588 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f499b659c-7p4tj"] Jul 20 15:28:54.067160 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:28:54.067129 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343c9bf5_baf3_4b4c_9255_8ddc7c560026.slice/crio-322d11ff0c145d5ae7bc211ee2eb630e461fb07007f36b0804058f63e8e25804 WatchSource:0}: Error finding container 322d11ff0c145d5ae7bc211ee2eb630e461fb07007f36b0804058f63e8e25804: Status 404 returned error can't find the container with id 322d11ff0c145d5ae7bc211ee2eb630e461fb07007f36b0804058f63e8e25804 Jul 20 15:28:54.209876 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:54.209798 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" event={"ID":"343c9bf5-baf3-4b4c-9255-8ddc7c560026","Type":"ContainerStarted","Data":"322d11ff0c145d5ae7bc211ee2eb630e461fb07007f36b0804058f63e8e25804"} Jul 20 15:28:57.224077 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.224041 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" event={"ID":"343c9bf5-baf3-4b4c-9255-8ddc7c560026","Type":"ContainerStarted","Data":"157f86ece6b4d0724ec882c118c907989742b1aae965655d10a7d9f1db1f99e3"} Jul 20 15:28:57.224447 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.224098 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" event={"ID":"343c9bf5-baf3-4b4c-9255-8ddc7c560026","Type":"ContainerStarted","Data":"a99a7aa792591cbf511d97053e4a6dbac7e550bf9973e0d410e9dbf13015b4d8"} Jul 20 15:28:57.224447 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.224117 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" event={"ID":"343c9bf5-baf3-4b4c-9255-8ddc7c560026","Type":"ContainerStarted","Data":"3d2c750f907ce5c1c6ba4d1c335c1165004e53d8f8d54730033a1697d4ba6b8a"} Jul 20 15:28:57.247081 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.247036 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f499b659c-7p4tj" podStartSLOduration=1.991647666 podStartE2EDuration="4.24702223s" totalImagesPullingTime="2.255374564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:57.246470685 +0000 UTC m=+129.191064066" watchObservedRunningTime="2026-07-20 15:28:57.24702223 +0000 UTC m=+129.191615599" Jul 20 15:28:57.921808 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.921760 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-88c7b66b6-b7f2b"] Jul 20 15:28:57.925134 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.925112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.936142 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.936120 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-88c7b66b6-b7f2b"] Jul 20 15:28:57.961540 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-oauth-serving-cert\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.961730 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-trusted-ca-bundle\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.961730 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961586 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-oauth-config\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.961730 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961609 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-service-ca\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.961730 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961685 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-console-config\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.961730 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961724 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw8h9\" (UniqueName: \"kubernetes.io/projected/91c7e13a-410b-460e-aded-ad60a00b512d-kube-api-access-zw8h9\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:57.962003 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:57.961748 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-serving-cert\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062555 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw8h9\" (UniqueName: \"kubernetes.io/projected/91c7e13a-410b-460e-aded-ad60a00b512d-kube-api-access-zw8h9\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062555 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062559 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-serving-cert\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062759 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-oauth-serving-cert\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062759 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062728 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-trusted-ca-bundle\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062894 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062764 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-oauth-config\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062894 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062811 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-service-ca\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.062987 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.062915 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-console-config\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.063539 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.063489 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-oauth-serving-cert\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.063664 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.063602 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-service-ca\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.063719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.063663 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-console-config\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.063719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.063674 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-trusted-ca-bundle\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.065318 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.065287 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-oauth-config\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.065675 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.065653 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-serving-cert\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.070042 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.070022 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw8h9\" (UniqueName: \"kubernetes.io/projected/91c7e13a-410b-460e-aded-ad60a00b512d-kube-api-access-zw8h9\") pod \"console-88c7b66b6-b7f2b\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.233699 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.233632 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:28:58.361697 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:58.361669 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-88c7b66b6-b7f2b"] Jul 20 15:28:58.364298 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:28:58.364267 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c7e13a_410b_460e_aded_ad60a00b512d.slice/crio-fae0f6062b9f1bb742b404c82755cb588617fcf75c9073e132cd930f79a9316f WatchSource:0}: Error finding container fae0f6062b9f1bb742b404c82755cb588617fcf75c9073e132cd930f79a9316f: Status 404 returned error can't find the container with id fae0f6062b9f1bb742b404c82755cb588617fcf75c9073e132cd930f79a9316f Jul 20 15:28:59.232564 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:59.232523 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88c7b66b6-b7f2b" event={"ID":"91c7e13a-410b-460e-aded-ad60a00b512d","Type":"ContainerStarted","Data":"dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133"} Jul 20 15:28:59.232701 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:59.232609 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88c7b66b6-b7f2b" event={"ID":"91c7e13a-410b-460e-aded-ad60a00b512d","Type":"ContainerStarted","Data":"fae0f6062b9f1bb742b404c82755cb588617fcf75c9073e132cd930f79a9316f"} Jul 20 15:28:59.254714 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:28:59.254664 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-88c7b66b6-b7f2b" podStartSLOduration=2.25464924 podStartE2EDuration="2.25464924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:28:59.252818011 +0000 UTC m=+131.197411381" watchObservedRunningTime="2026-07-20 15:28:59.25464924 +0000 UTC m=+131.199242610" Jul 20 15:29:08.234162 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.234122 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:29:08.234721 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.234214 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:29:08.239203 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.239180 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:29:08.268911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.268890 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:29:08.316421 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.316396 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d75d484c-jtqw2"] Jul 20 15:29:08.441620 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.441587 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b8959f7c6-47kcb"] Jul 20 15:29:08.444942 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.444925 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.459239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.459218 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b8959f7c6-47kcb"] Jul 20 15:29:08.541442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-oauth-config\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.541442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541401 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-trusted-ca-bundle\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.541635 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541464 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-serving-cert\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.541635 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541500 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-config\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.541635 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541526 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-service-ca\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.541635 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541558 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-oauth-serving-cert\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.541635 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.541590 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njbl8\" (UniqueName: \"kubernetes.io/projected/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-kube-api-access-njbl8\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.642743 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.642709 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-serving-cert\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.642924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.642792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-config\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.642924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.642829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-service-ca\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.642924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.642852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-oauth-serving-cert\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643063 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643001 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njbl8\" (UniqueName: \"kubernetes.io/projected/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-kube-api-access-njbl8\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643111 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-oauth-config\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643111 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-trusted-ca-bundle\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643592 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643562 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-config\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643705 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643631 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-service-ca\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643705 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-oauth-serving-cert\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.643882 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.643866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-trusted-ca-bundle\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.645469 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.645436 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-serving-cert\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.645618 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.645602 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-oauth-config\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.651642 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.651621 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njbl8\" (UniqueName: \"kubernetes.io/projected/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-kube-api-access-njbl8\") pod \"console-b8959f7c6-47kcb\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.753678 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.753646 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:08.903888 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:08.903859 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b8959f7c6-47kcb"] Jul 20 15:29:08.906634 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:29:08.906608 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc2449b_3b52_41a2_a8e5_e9970ba1794f.slice/crio-2c1649c832cd849e6b1e455f27085dc6984993cec1db37d8e4faf9c3527c160d WatchSource:0}: Error finding container 2c1649c832cd849e6b1e455f27085dc6984993cec1db37d8e4faf9c3527c160d: Status 404 returned error can't find the container with id 2c1649c832cd849e6b1e455f27085dc6984993cec1db37d8e4faf9c3527c160d Jul 20 15:29:09.270536 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:09.270498 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b8959f7c6-47kcb" event={"ID":"0dc2449b-3b52-41a2-a8e5-e9970ba1794f","Type":"ContainerStarted","Data":"e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab"} Jul 20 15:29:09.270920 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:09.270555 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b8959f7c6-47kcb" event={"ID":"0dc2449b-3b52-41a2-a8e5-e9970ba1794f","Type":"ContainerStarted","Data":"2c1649c832cd849e6b1e455f27085dc6984993cec1db37d8e4faf9c3527c160d"} Jul 20 15:29:09.289826 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:09.289756 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b8959f7c6-47kcb" podStartSLOduration=1.289739207 podStartE2EDuration="1.289739207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:29:09.289328606 +0000 UTC m=+141.233921976" watchObservedRunningTime="2026-07-20 15:29:09.289739207 +0000 UTC m=+141.234332577" Jul 20 15:29:18.754660 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:18.754608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:18.755161 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:18.754690 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:18.759475 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:18.759454 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:19.309244 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:19.309218 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:29:19.355090 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:19.355059 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-88c7b66b6-b7f2b"] Jul 20 15:29:33.336990 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.336926 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57d75d484c-jtqw2" podUID="6e9dc728-8b40-40c9-b778-d9005aef3c0f" containerName="console" containerID="cri-o://c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76" gracePeriod=15 Jul 20 15:29:33.577652 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.577630 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:29:33.739061 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739031 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-service-ca\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739236 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739073 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-serving-cert\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739236 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739105 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-trusted-ca-bundle\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739236 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739122 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-config\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739236 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739159 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-oauth-serving-cert\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739236 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739176 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-oauth-config\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739479 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739308 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4wxj\" (UniqueName: \"kubernetes.io/projected/6e9dc728-8b40-40c9-b778-d9005aef3c0f-kube-api-access-v4wxj\") pod \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\" (UID: \"6e9dc728-8b40-40c9-b778-d9005aef3c0f\") " Jul 20 15:29:33.739531 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739430 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:33.739613 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739595 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-service-ca\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:33.739663 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739637 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-config" (OuterVolumeSpecName: "console-config") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:33.739663 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739649 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:33.739728 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.739658 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:33.741487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.741455 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:29:33.741594 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.741560 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:29:33.741657 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.741612 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e9dc728-8b40-40c9-b778-d9005aef3c0f-kube-api-access-v4wxj" (OuterVolumeSpecName: "kube-api-access-v4wxj") pod "6e9dc728-8b40-40c9-b778-d9005aef3c0f" (UID: "6e9dc728-8b40-40c9-b778-d9005aef3c0f"). InnerVolumeSpecName "kube-api-access-v4wxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:29:33.840151 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.840117 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-oauth-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:33.840151 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.840146 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-oauth-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:33.840298 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.840158 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4wxj\" (UniqueName: \"kubernetes.io/projected/6e9dc728-8b40-40c9-b778-d9005aef3c0f-kube-api-access-v4wxj\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:33.840298 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.840170 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:33.840298 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.840181 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-trusted-ca-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:33.840298 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:33.840193 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e9dc728-8b40-40c9-b778-d9005aef3c0f-console-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:34.361852 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.361817 2557 generic.go:356] "Generic (PLEG): container finished" podID="6e9dc728-8b40-40c9-b778-d9005aef3c0f" containerID="c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76" exitCode=0 Jul 20 15:29:34.362241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.361861 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75d484c-jtqw2" Jul 20 15:29:34.362241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.361901 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75d484c-jtqw2" event={"ID":"6e9dc728-8b40-40c9-b778-d9005aef3c0f","Type":"ContainerDied","Data":"c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76"} Jul 20 15:29:34.362241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.361956 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75d484c-jtqw2" event={"ID":"6e9dc728-8b40-40c9-b778-d9005aef3c0f","Type":"ContainerDied","Data":"0cea90a553fce26c2652b051b0a4bf2ce5984c9ecb860093a131f4d147c94c82"} Jul 20 15:29:34.362241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.361979 2557 scope.go:117] "RemoveContainer" containerID="c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76" Jul 20 15:29:34.373482 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.373297 2557 scope.go:117] "RemoveContainer" containerID="c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76" Jul 20 15:29:34.373936 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:29:34.373906 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76\": container with ID starting with c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76 not found: ID does not exist" containerID="c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76" Jul 20 15:29:34.374033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.373943 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76"} err="failed to get container status \"c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76\": rpc error: code = NotFound desc = could not find container \"c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76\": container with ID starting with c8cd70e1f64eae302095a325883c07ef962d5d0ed0401905b6a3a39ae2202d76 not found: ID does not exist" Jul 20 15:29:34.386103 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.386082 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d75d484c-jtqw2"] Jul 20 15:29:34.390220 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.390202 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57d75d484c-jtqw2"] Jul 20 15:29:34.564055 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:34.564024 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e9dc728-8b40-40c9-b778-d9005aef3c0f" path="/var/lib/kubelet/pods/6e9dc728-8b40-40c9-b778-d9005aef3c0f/volumes" Jul 20 15:29:44.377081 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.377017 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-88c7b66b6-b7f2b" podUID="91c7e13a-410b-460e-aded-ad60a00b512d" containerName="console" containerID="cri-o://dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133" gracePeriod=15 Jul 20 15:29:44.626611 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.626588 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:29:44.813582 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813490 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-serving-cert\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.813582 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813564 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-oauth-config\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.813582 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813588 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-trusted-ca-bundle\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.813863 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813613 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-oauth-serving-cert\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.813863 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813636 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-service-ca\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.813863 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813659 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-console-config\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.813863 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.813752 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw8h9\" (UniqueName: \"kubernetes.io/projected/91c7e13a-410b-460e-aded-ad60a00b512d-kube-api-access-zw8h9\") pod \"91c7e13a-410b-460e-aded-ad60a00b512d\" (UID: \"91c7e13a-410b-460e-aded-ad60a00b512d\") " Jul 20 15:29:44.814123 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.814092 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-console-config" (OuterVolumeSpecName: "console-config") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:44.814226 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.814144 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:44.814226 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.814148 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:44.814226 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.814218 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-service-ca" (OuterVolumeSpecName: "service-ca") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:29:44.815751 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.815720 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:29:44.815851 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.815790 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:29:44.816021 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.816001 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c7e13a-410b-460e-aded-ad60a00b512d-kube-api-access-zw8h9" (OuterVolumeSpecName: "kube-api-access-zw8h9") pod "91c7e13a-410b-460e-aded-ad60a00b512d" (UID: "91c7e13a-410b-460e-aded-ad60a00b512d"). InnerVolumeSpecName "kube-api-access-zw8h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:29:44.914365 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914337 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:44.914365 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914362 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91c7e13a-410b-460e-aded-ad60a00b512d-console-oauth-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:44.914516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914375 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-trusted-ca-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:44.914516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914387 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-oauth-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:44.914516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914400 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-service-ca\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:44.914516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914412 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91c7e13a-410b-460e-aded-ad60a00b512d-console-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:44.914516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:44.914423 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw8h9\" (UniqueName: \"kubernetes.io/projected/91c7e13a-410b-460e-aded-ad60a00b512d-kube-api-access-zw8h9\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:29:45.407610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.407571 2557 generic.go:356] "Generic (PLEG): container finished" podID="91c7e13a-410b-460e-aded-ad60a00b512d" containerID="dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133" exitCode=0 Jul 20 15:29:45.407998 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.407638 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88c7b66b6-b7f2b" Jul 20 15:29:45.407998 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.407657 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88c7b66b6-b7f2b" event={"ID":"91c7e13a-410b-460e-aded-ad60a00b512d","Type":"ContainerDied","Data":"dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133"} Jul 20 15:29:45.407998 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.407736 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88c7b66b6-b7f2b" event={"ID":"91c7e13a-410b-460e-aded-ad60a00b512d","Type":"ContainerDied","Data":"fae0f6062b9f1bb742b404c82755cb588617fcf75c9073e132cd930f79a9316f"} Jul 20 15:29:45.407998 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.407762 2557 scope.go:117] "RemoveContainer" containerID="dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133" Jul 20 15:29:45.416028 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.416011 2557 scope.go:117] "RemoveContainer" containerID="dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133" Jul 20 15:29:45.416296 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:29:45.416276 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133\": container with ID starting with dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133 not found: ID does not exist" containerID="dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133" Jul 20 15:29:45.416354 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.416305 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133"} err="failed to get container status \"dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133\": rpc error: code = NotFound desc = could not find container \"dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133\": container with ID starting with dcbd54a04bd4ea5f3c8c541ac6b4e9d7de716865ba450a290b5d1140a5096133 not found: ID does not exist" Jul 20 15:29:45.432924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.432904 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-88c7b66b6-b7f2b"] Jul 20 15:29:45.439674 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:45.439653 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-88c7b66b6-b7f2b"] Jul 20 15:29:46.563706 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:29:46.563664 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91c7e13a-410b-460e-aded-ad60a00b512d" path="/var/lib/kubelet/pods/91c7e13a-410b-460e-aded-ad60a00b512d/volumes" Jul 20 15:30:04.557618 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557587 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k"] Jul 20 15:30:04.558009 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557889 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e9dc728-8b40-40c9-b778-d9005aef3c0f" containerName="console" Jul 20 15:30:04.558009 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557900 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e9dc728-8b40-40c9-b778-d9005aef3c0f" containerName="console" Jul 20 15:30:04.558009 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557916 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91c7e13a-410b-460e-aded-ad60a00b512d" containerName="console" Jul 20 15:30:04.558009 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557922 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="91c7e13a-410b-460e-aded-ad60a00b512d" containerName="console" Jul 20 15:30:04.558009 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557971 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e9dc728-8b40-40c9-b778-d9005aef3c0f" containerName="console" Jul 20 15:30:04.558009 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.557978 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="91c7e13a-410b-460e-aded-ad60a00b512d" containerName="console" Jul 20 15:30:04.561145 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.561123 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.563617 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.563594 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7fjhx\"" Jul 20 15:30:04.564144 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.564113 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 15:30:04.564279 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.564172 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 15:30:04.574038 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.574017 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k"] Jul 20 15:30:04.660627 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.660588 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.660825 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.660668 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.660825 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.660721 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jk78\" (UniqueName: \"kubernetes.io/projected/d41e4cd9-0563-4de4-a83e-5567803af975-kube-api-access-2jk78\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.761901 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.761867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.762036 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.761912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.762036 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.761941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jk78\" (UniqueName: \"kubernetes.io/projected/d41e4cd9-0563-4de4-a83e-5567803af975-kube-api-access-2jk78\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.762240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.762223 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.762280 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.762249 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.772907 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.772880 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jk78\" (UniqueName: \"kubernetes.io/projected/d41e4cd9-0563-4de4-a83e-5567803af975-kube-api-access-2jk78\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:04.871216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:04.871144 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:05.002492 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:05.002460 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k"] Jul 20 15:30:05.005944 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:30:05.005919 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41e4cd9_0563_4de4_a83e_5567803af975.slice/crio-745a0850cccad6ac502d53a603c68ec8b755586c1b87ffd9a8d6c0e3d09a80d0 WatchSource:0}: Error finding container 745a0850cccad6ac502d53a603c68ec8b755586c1b87ffd9a8d6c0e3d09a80d0: Status 404 returned error can't find the container with id 745a0850cccad6ac502d53a603c68ec8b755586c1b87ffd9a8d6c0e3d09a80d0 Jul 20 15:30:05.479354 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:05.479313 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" event={"ID":"d41e4cd9-0563-4de4-a83e-5567803af975","Type":"ContainerStarted","Data":"745a0850cccad6ac502d53a603c68ec8b755586c1b87ffd9a8d6c0e3d09a80d0"} Jul 20 15:30:12.506387 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:12.506348 2557 generic.go:356] "Generic (PLEG): container finished" podID="d41e4cd9-0563-4de4-a83e-5567803af975" containerID="7444f06698a37dbe5ecc2d9b8caadad72ba94db8e1403e7fb3cf79f466de208e" exitCode=0 Jul 20 15:30:12.506910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:12.506407 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" event={"ID":"d41e4cd9-0563-4de4-a83e-5567803af975","Type":"ContainerDied","Data":"7444f06698a37dbe5ecc2d9b8caadad72ba94db8e1403e7fb3cf79f466de208e"} Jul 20 15:30:29.565368 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:29.565282 2557 generic.go:356] "Generic (PLEG): container finished" podID="d41e4cd9-0563-4de4-a83e-5567803af975" containerID="d90151f3f088de7e5e690a51accc8d787238bb8df3bcbf1cc36a3c00caeeb4b6" exitCode=0 Jul 20 15:30:29.565790 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:29.565373 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" event={"ID":"d41e4cd9-0563-4de4-a83e-5567803af975","Type":"ContainerDied","Data":"d90151f3f088de7e5e690a51accc8d787238bb8df3bcbf1cc36a3c00caeeb4b6"} Jul 20 15:30:37.598465 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:37.598426 2557 generic.go:356] "Generic (PLEG): container finished" podID="d41e4cd9-0563-4de4-a83e-5567803af975" containerID="68ea556ee7e2fa9e3eece2d412130274e7d8c260d85612d35dc0fa03d6f27642" exitCode=0 Jul 20 15:30:37.598851 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:37.598463 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" event={"ID":"d41e4cd9-0563-4de4-a83e-5567803af975","Type":"ContainerDied","Data":"68ea556ee7e2fa9e3eece2d412130274e7d8c260d85612d35dc0fa03d6f27642"} Jul 20 15:30:38.724006 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.723985 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:38.841575 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.841535 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jk78\" (UniqueName: \"kubernetes.io/projected/d41e4cd9-0563-4de4-a83e-5567803af975-kube-api-access-2jk78\") pod \"d41e4cd9-0563-4de4-a83e-5567803af975\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " Jul 20 15:30:38.841727 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.841586 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-util\") pod \"d41e4cd9-0563-4de4-a83e-5567803af975\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " Jul 20 15:30:38.841727 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.841637 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-bundle\") pod \"d41e4cd9-0563-4de4-a83e-5567803af975\" (UID: \"d41e4cd9-0563-4de4-a83e-5567803af975\") " Jul 20 15:30:38.842366 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.842334 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-bundle" (OuterVolumeSpecName: "bundle") pod "d41e4cd9-0563-4de4-a83e-5567803af975" (UID: "d41e4cd9-0563-4de4-a83e-5567803af975"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:30:38.843750 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.843724 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41e4cd9-0563-4de4-a83e-5567803af975-kube-api-access-2jk78" (OuterVolumeSpecName: "kube-api-access-2jk78") pod "d41e4cd9-0563-4de4-a83e-5567803af975" (UID: "d41e4cd9-0563-4de4-a83e-5567803af975"). InnerVolumeSpecName "kube-api-access-2jk78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:30:38.845406 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.845389 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-util" (OuterVolumeSpecName: "util") pod "d41e4cd9-0563-4de4-a83e-5567803af975" (UID: "d41e4cd9-0563-4de4-a83e-5567803af975"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:30:38.942408 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.942379 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:30:38.942408 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.942400 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41e4cd9-0563-4de4-a83e-5567803af975-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:30:38.942408 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:38.942410 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jk78\" (UniqueName: \"kubernetes.io/projected/d41e4cd9-0563-4de4-a83e-5567803af975-kube-api-access-2jk78\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:30:39.607548 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:39.607514 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" event={"ID":"d41e4cd9-0563-4de4-a83e-5567803af975","Type":"ContainerDied","Data":"745a0850cccad6ac502d53a603c68ec8b755586c1b87ffd9a8d6c0e3d09a80d0"} Jul 20 15:30:39.607717 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:39.607555 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409lb6k" Jul 20 15:30:39.607717 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:39.607565 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="745a0850cccad6ac502d53a603c68ec8b755586c1b87ffd9a8d6c0e3d09a80d0" Jul 20 15:30:43.526514 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526476 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7"] Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526895 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="util" Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526912 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="util" Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526924 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="pull" Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526931 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="pull" Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526949 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="extract" Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.526957 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="extract" Jul 20 15:30:43.527108 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.527044 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41e4cd9-0563-4de4-a83e-5567803af975" containerName="extract" Jul 20 15:30:43.530318 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.530292 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.532330 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.532309 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 20 15:30:43.532444 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.532331 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zfwtm\"" Jul 20 15:30:43.532718 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.532704 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:30:43.555910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.555883 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7"] Jul 20 15:30:43.575986 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.575946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxkvq\" (UniqueName: \"kubernetes.io/projected/39e892f9-514e-4e28-a1f8-1c9990fcccc8-kube-api-access-mxkvq\") pod \"cert-manager-operator-controller-manager-55c444bf86-f8sd7\" (UID: \"39e892f9-514e-4e28-a1f8-1c9990fcccc8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.576126 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.576074 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39e892f9-514e-4e28-a1f8-1c9990fcccc8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-f8sd7\" (UID: \"39e892f9-514e-4e28-a1f8-1c9990fcccc8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.676796 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.676735 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxkvq\" (UniqueName: \"kubernetes.io/projected/39e892f9-514e-4e28-a1f8-1c9990fcccc8-kube-api-access-mxkvq\") pod \"cert-manager-operator-controller-manager-55c444bf86-f8sd7\" (UID: \"39e892f9-514e-4e28-a1f8-1c9990fcccc8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.676796 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.676795 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39e892f9-514e-4e28-a1f8-1c9990fcccc8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-f8sd7\" (UID: \"39e892f9-514e-4e28-a1f8-1c9990fcccc8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.677243 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.677224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39e892f9-514e-4e28-a1f8-1c9990fcccc8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-f8sd7\" (UID: \"39e892f9-514e-4e28-a1f8-1c9990fcccc8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.688209 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.688185 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxkvq\" (UniqueName: \"kubernetes.io/projected/39e892f9-514e-4e28-a1f8-1c9990fcccc8-kube-api-access-mxkvq\") pod \"cert-manager-operator-controller-manager-55c444bf86-f8sd7\" (UID: \"39e892f9-514e-4e28-a1f8-1c9990fcccc8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.839240 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.839152 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" Jul 20 15:30:43.979499 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:43.979475 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7"] Jul 20 15:30:43.982337 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:30:43.982311 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e892f9_514e_4e28_a1f8_1c9990fcccc8.slice/crio-0022771366f117aafbebaa70cac38362bbdf4a64c6501d7356ee34217261d1bb WatchSource:0}: Error finding container 0022771366f117aafbebaa70cac38362bbdf4a64c6501d7356ee34217261d1bb: Status 404 returned error can't find the container with id 0022771366f117aafbebaa70cac38362bbdf4a64c6501d7356ee34217261d1bb Jul 20 15:30:44.631487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:44.631444 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" event={"ID":"39e892f9-514e-4e28-a1f8-1c9990fcccc8","Type":"ContainerStarted","Data":"0022771366f117aafbebaa70cac38362bbdf4a64c6501d7356ee34217261d1bb"} Jul 20 15:30:46.640362 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:46.640330 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" event={"ID":"39e892f9-514e-4e28-a1f8-1c9990fcccc8","Type":"ContainerStarted","Data":"f9870b0f164ee6c933ffa784bec0f5e9d634c143961aef36c8c780f875292e03"} Jul 20 15:30:46.684864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:46.684820 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f8sd7" podStartSLOduration=1.988014905 podStartE2EDuration="3.684805239s" totalImagesPullingTime="1.696790334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:30:46.679702259 +0000 UTC m=+238.624295628" watchObservedRunningTime="2026-07-20 15:30:46.684805239 +0000 UTC m=+238.629398610" Jul 20 15:30:47.026796 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.026694 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xnsq2"] Jul 20 15:30:47.029956 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.029940 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.036507 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.036370 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 15:30:47.036623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.036592 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 15:30:47.036742 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.036723 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vvnmb\"" Jul 20 15:30:47.058217 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.058190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xnsq2"] Jul 20 15:30:47.103680 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.103647 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/443a2025-a4c7-459a-bfd6-cb077bce22d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xnsq2\" (UID: \"443a2025-a4c7-459a-bfd6-cb077bce22d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.103840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.103715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86hlt\" (UniqueName: \"kubernetes.io/projected/443a2025-a4c7-459a-bfd6-cb077bce22d1-kube-api-access-86hlt\") pod \"cert-manager-webhook-6486ffd997-xnsq2\" (UID: \"443a2025-a4c7-459a-bfd6-cb077bce22d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.204990 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.204952 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/443a2025-a4c7-459a-bfd6-cb077bce22d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xnsq2\" (UID: \"443a2025-a4c7-459a-bfd6-cb077bce22d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.205135 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.205051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86hlt\" (UniqueName: \"kubernetes.io/projected/443a2025-a4c7-459a-bfd6-cb077bce22d1-kube-api-access-86hlt\") pod \"cert-manager-webhook-6486ffd997-xnsq2\" (UID: \"443a2025-a4c7-459a-bfd6-cb077bce22d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.232620 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.232587 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86hlt\" (UniqueName: \"kubernetes.io/projected/443a2025-a4c7-459a-bfd6-cb077bce22d1-kube-api-access-86hlt\") pod \"cert-manager-webhook-6486ffd997-xnsq2\" (UID: \"443a2025-a4c7-459a-bfd6-cb077bce22d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.234310 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.234284 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/443a2025-a4c7-459a-bfd6-cb077bce22d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xnsq2\" (UID: \"443a2025-a4c7-459a-bfd6-cb077bce22d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.347438 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.347348 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:47.499961 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.499930 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xnsq2"] Jul 20 15:30:47.503074 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:30:47.503049 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod443a2025_a4c7_459a_bfd6_cb077bce22d1.slice/crio-d8a84e74e04b7f7f0ac920e18b5c9299e0ec7ca3a0c488bdd715389e733c249e WatchSource:0}: Error finding container d8a84e74e04b7f7f0ac920e18b5c9299e0ec7ca3a0c488bdd715389e733c249e: Status 404 returned error can't find the container with id d8a84e74e04b7f7f0ac920e18b5c9299e0ec7ca3a0c488bdd715389e733c249e Jul 20 15:30:47.534115 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.534083 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f"] Jul 20 15:30:47.538658 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.538641 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.559659 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.559641 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 15:30:47.559797 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.559639 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 15:30:47.582482 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.582461 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7fjhx\"" Jul 20 15:30:47.590079 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.590050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f"] Jul 20 15:30:47.608903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.608852 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.608903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.608880 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fscn\" (UniqueName: \"kubernetes.io/projected/7e12addf-27f5-46b6-9f97-11d888383362-kube-api-access-8fscn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.609031 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.608916 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.645254 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.645226 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" event={"ID":"443a2025-a4c7-459a-bfd6-cb077bce22d1","Type":"ContainerStarted","Data":"d8a84e74e04b7f7f0ac920e18b5c9299e0ec7ca3a0c488bdd715389e733c249e"} Jul 20 15:30:47.709467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.709438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.709467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.709469 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fscn\" (UniqueName: \"kubernetes.io/projected/7e12addf-27f5-46b6-9f97-11d888383362-kube-api-access-8fscn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.709627 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.709499 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.709903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.709885 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.709942 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.709895 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.730547 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.730524 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fscn\" (UniqueName: \"kubernetes.io/projected/7e12addf-27f5-46b6-9f97-11d888383362-kube-api-access-8fscn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.847304 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.847279 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:47.997664 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:47.997630 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f"] Jul 20 15:30:48.002954 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:30:48.002925 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e12addf_27f5_46b6_9f97_11d888383362.slice/crio-37d18b587ac2ba8199e49cfe9c304d8c5e6e73022d11e4e68cf785104b00b44d WatchSource:0}: Error finding container 37d18b587ac2ba8199e49cfe9c304d8c5e6e73022d11e4e68cf785104b00b44d: Status 404 returned error can't find the container with id 37d18b587ac2ba8199e49cfe9c304d8c5e6e73022d11e4e68cf785104b00b44d Jul 20 15:30:48.651920 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:48.651881 2557 generic.go:356] "Generic (PLEG): container finished" podID="7e12addf-27f5-46b6-9f97-11d888383362" containerID="75ec21f51fd1e36458bf058b3bff91b5eb693521b510358cc189da809095e575" exitCode=0 Jul 20 15:30:48.652341 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:48.651972 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" event={"ID":"7e12addf-27f5-46b6-9f97-11d888383362","Type":"ContainerDied","Data":"75ec21f51fd1e36458bf058b3bff91b5eb693521b510358cc189da809095e575"} Jul 20 15:30:48.652341 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:48.652038 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" event={"ID":"7e12addf-27f5-46b6-9f97-11d888383362","Type":"ContainerStarted","Data":"37d18b587ac2ba8199e49cfe9c304d8c5e6e73022d11e4e68cf785104b00b44d"} Jul 20 15:30:51.669213 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:51.669180 2557 generic.go:356] "Generic (PLEG): container finished" podID="7e12addf-27f5-46b6-9f97-11d888383362" containerID="32596213288aed01cf1749fe36fa204ef56c041e00be8a4ac5bca817ff3fe61e" exitCode=0 Jul 20 15:30:51.669678 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:51.669268 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" event={"ID":"7e12addf-27f5-46b6-9f97-11d888383362","Type":"ContainerDied","Data":"32596213288aed01cf1749fe36fa204ef56c041e00be8a4ac5bca817ff3fe61e"} Jul 20 15:30:51.670954 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:51.670929 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" event={"ID":"443a2025-a4c7-459a-bfd6-cb077bce22d1","Type":"ContainerStarted","Data":"5d2be4790162edd818e8e04d23b295e888e841d972c8bbad858b07247ed25017"} Jul 20 15:30:51.671042 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:51.671012 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:51.736325 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:51.736273 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" podStartSLOduration=2.211142945 podStartE2EDuration="5.736252482s" totalImagesPullingTime="3.525109537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:30:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:30:51.734430691 +0000 UTC m=+243.679024062" watchObservedRunningTime="2026-07-20 15:30:51.736252482 +0000 UTC m=+243.680845852" Jul 20 15:30:52.677239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:52.677204 2557 generic.go:356] "Generic (PLEG): container finished" podID="7e12addf-27f5-46b6-9f97-11d888383362" containerID="ce682f15df2c4552eb1987eb7818a8ab414a53702b729a9be64e5446ae73521e" exitCode=0 Jul 20 15:30:52.677659 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:52.677289 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" event={"ID":"7e12addf-27f5-46b6-9f97-11d888383362","Type":"ContainerDied","Data":"ce682f15df2c4552eb1987eb7818a8ab414a53702b729a9be64e5446ae73521e"} Jul 20 15:30:53.358480 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.358441 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2"] Jul 20 15:30:53.362173 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.362150 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.364888 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.364865 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8nfnd\"" Jul 20 15:30:53.374092 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.374058 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2"] Jul 20 15:30:53.456915 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.456872 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmtpr\" (UniqueName: \"kubernetes.io/projected/0aafbf7b-0017-4983-93d5-bfe8e41b3b10-kube-api-access-hmtpr\") pod \"cert-manager-cainjector-559cdd8c87-jdzs2\" (UID: \"0aafbf7b-0017-4983-93d5-bfe8e41b3b10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.456915 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.456918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aafbf7b-0017-4983-93d5-bfe8e41b3b10-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jdzs2\" (UID: \"0aafbf7b-0017-4983-93d5-bfe8e41b3b10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.558355 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.558312 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmtpr\" (UniqueName: \"kubernetes.io/projected/0aafbf7b-0017-4983-93d5-bfe8e41b3b10-kube-api-access-hmtpr\") pod \"cert-manager-cainjector-559cdd8c87-jdzs2\" (UID: \"0aafbf7b-0017-4983-93d5-bfe8e41b3b10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.558519 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.558359 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aafbf7b-0017-4983-93d5-bfe8e41b3b10-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jdzs2\" (UID: \"0aafbf7b-0017-4983-93d5-bfe8e41b3b10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.579084 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.579048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmtpr\" (UniqueName: \"kubernetes.io/projected/0aafbf7b-0017-4983-93d5-bfe8e41b3b10-kube-api-access-hmtpr\") pod \"cert-manager-cainjector-559cdd8c87-jdzs2\" (UID: \"0aafbf7b-0017-4983-93d5-bfe8e41b3b10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.583784 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.583746 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0aafbf7b-0017-4983-93d5-bfe8e41b3b10-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jdzs2\" (UID: \"0aafbf7b-0017-4983-93d5-bfe8e41b3b10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.671013 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.670980 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" Jul 20 15:30:53.824234 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.824206 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2"] Jul 20 15:30:53.826327 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.826310 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:53.828152 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:30:53.828125 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aafbf7b_0017_4983_93d5_bfe8e41b3b10.slice/crio-bda9dc944e5ca6d241aac8e951d0a770e0428dce5a55ec235509d4c148b56521 WatchSource:0}: Error finding container bda9dc944e5ca6d241aac8e951d0a770e0428dce5a55ec235509d4c148b56521: Status 404 returned error can't find the container with id bda9dc944e5ca6d241aac8e951d0a770e0428dce5a55ec235509d4c148b56521 Jul 20 15:30:53.860908 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.860878 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fscn\" (UniqueName: \"kubernetes.io/projected/7e12addf-27f5-46b6-9f97-11d888383362-kube-api-access-8fscn\") pod \"7e12addf-27f5-46b6-9f97-11d888383362\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " Jul 20 15:30:53.861045 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.860922 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-bundle\") pod \"7e12addf-27f5-46b6-9f97-11d888383362\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " Jul 20 15:30:53.861045 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.860959 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-util\") pod \"7e12addf-27f5-46b6-9f97-11d888383362\" (UID: \"7e12addf-27f5-46b6-9f97-11d888383362\") " Jul 20 15:30:53.861579 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.861408 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-bundle" (OuterVolumeSpecName: "bundle") pod "7e12addf-27f5-46b6-9f97-11d888383362" (UID: "7e12addf-27f5-46b6-9f97-11d888383362"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:30:53.863120 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.863090 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e12addf-27f5-46b6-9f97-11d888383362-kube-api-access-8fscn" (OuterVolumeSpecName: "kube-api-access-8fscn") pod "7e12addf-27f5-46b6-9f97-11d888383362" (UID: "7e12addf-27f5-46b6-9f97-11d888383362"). InnerVolumeSpecName "kube-api-access-8fscn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:30:53.866499 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.866464 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-util" (OuterVolumeSpecName: "util") pod "7e12addf-27f5-46b6-9f97-11d888383362" (UID: "7e12addf-27f5-46b6-9f97-11d888383362"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:30:53.961801 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.961703 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fscn\" (UniqueName: \"kubernetes.io/projected/7e12addf-27f5-46b6-9f97-11d888383362-kube-api-access-8fscn\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:30:53.961801 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.961736 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:30:53.961801 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:53.961750 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e12addf-27f5-46b6-9f97-11d888383362-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:30:54.691837 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:54.691792 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" event={"ID":"0aafbf7b-0017-4983-93d5-bfe8e41b3b10","Type":"ContainerStarted","Data":"12502dc1bcdddc17b915ffd7a5ed3bffa7484807da5cba5b629d0d4a1595ea35"} Jul 20 15:30:54.692011 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:54.691849 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" event={"ID":"0aafbf7b-0017-4983-93d5-bfe8e41b3b10","Type":"ContainerStarted","Data":"bda9dc944e5ca6d241aac8e951d0a770e0428dce5a55ec235509d4c148b56521"} Jul 20 15:30:54.693639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:54.693614 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" event={"ID":"7e12addf-27f5-46b6-9f97-11d888383362","Type":"ContainerDied","Data":"37d18b587ac2ba8199e49cfe9c304d8c5e6e73022d11e4e68cf785104b00b44d"} Jul 20 15:30:54.693639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:54.693634 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f49p6f" Jul 20 15:30:54.693820 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:54.693680 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37d18b587ac2ba8199e49cfe9c304d8c5e6e73022d11e4e68cf785104b00b44d" Jul 20 15:30:54.718413 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:54.718371 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jdzs2" podStartSLOduration=1.718357761 podStartE2EDuration="1.718357761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:30:54.717336747 +0000 UTC m=+246.661930118" watchObservedRunningTime="2026-07-20 15:30:54.718357761 +0000 UTC m=+246.662951133" Jul 20 15:30:56.939540 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939508 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5fs92"] Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939808 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="util" Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939824 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="util" Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939832 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="extract" Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939838 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="extract" Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939858 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="pull" Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939863 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="pull" Jul 20 15:30:56.939918 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.939912 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e12addf-27f5-46b6-9f97-11d888383362" containerName="extract" Jul 20 15:30:56.943420 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.943404 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:56.950435 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.950416 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9ltz8\"" Jul 20 15:30:56.967690 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.967668 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5fs92"] Jul 20 15:30:56.986289 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.986263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab0b1e3a-0d8d-4be5-9356-7a214aa26689-bound-sa-token\") pod \"cert-manager-55b4d954d5-5fs92\" (UID: \"ab0b1e3a-0d8d-4be5-9356-7a214aa26689\") " pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:56.986377 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:56.986308 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dnn4\" (UniqueName: \"kubernetes.io/projected/ab0b1e3a-0d8d-4be5-9356-7a214aa26689-kube-api-access-9dnn4\") pod \"cert-manager-55b4d954d5-5fs92\" (UID: \"ab0b1e3a-0d8d-4be5-9356-7a214aa26689\") " pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:57.086874 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.086849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab0b1e3a-0d8d-4be5-9356-7a214aa26689-bound-sa-token\") pod \"cert-manager-55b4d954d5-5fs92\" (UID: \"ab0b1e3a-0d8d-4be5-9356-7a214aa26689\") " pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:57.087034 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.086884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dnn4\" (UniqueName: \"kubernetes.io/projected/ab0b1e3a-0d8d-4be5-9356-7a214aa26689-kube-api-access-9dnn4\") pod \"cert-manager-55b4d954d5-5fs92\" (UID: \"ab0b1e3a-0d8d-4be5-9356-7a214aa26689\") " pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:57.102909 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.102876 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab0b1e3a-0d8d-4be5-9356-7a214aa26689-bound-sa-token\") pod \"cert-manager-55b4d954d5-5fs92\" (UID: \"ab0b1e3a-0d8d-4be5-9356-7a214aa26689\") " pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:57.102994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.102970 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dnn4\" (UniqueName: \"kubernetes.io/projected/ab0b1e3a-0d8d-4be5-9356-7a214aa26689-kube-api-access-9dnn4\") pod \"cert-manager-55b4d954d5-5fs92\" (UID: \"ab0b1e3a-0d8d-4be5-9356-7a214aa26689\") " pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:57.251554 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.251498 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5fs92" Jul 20 15:30:57.379925 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.379785 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5fs92"] Jul 20 15:30:57.382402 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:30:57.382372 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab0b1e3a_0d8d_4be5_9356_7a214aa26689.slice/crio-35d24f18fe4999752d7ca7eaf72392ad70c8b23cd757d0ab75eaad822ac3ace4 WatchSource:0}: Error finding container 35d24f18fe4999752d7ca7eaf72392ad70c8b23cd757d0ab75eaad822ac3ace4: Status 404 returned error can't find the container with id 35d24f18fe4999752d7ca7eaf72392ad70c8b23cd757d0ab75eaad822ac3ace4 Jul 20 15:30:57.680016 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.679989 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xnsq2" Jul 20 15:30:57.707888 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.707851 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5fs92" event={"ID":"ab0b1e3a-0d8d-4be5-9356-7a214aa26689","Type":"ContainerStarted","Data":"cf6c87ce27008c5967e1854bd2dcfca044a01524b179850332176ba2b9ab8025"} Jul 20 15:30:57.708010 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.707891 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5fs92" event={"ID":"ab0b1e3a-0d8d-4be5-9356-7a214aa26689","Type":"ContainerStarted","Data":"35d24f18fe4999752d7ca7eaf72392ad70c8b23cd757d0ab75eaad822ac3ace4"} Jul 20 15:30:57.729306 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:30:57.729263 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5fs92" podStartSLOduration=1.729247258 podStartE2EDuration="1.729247258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:30:57.7280445 +0000 UTC m=+249.672637869" watchObservedRunningTime="2026-07-20 15:30:57.729247258 +0000 UTC m=+249.673840628" Jul 20 15:31:04.287970 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.287936 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4"] Jul 20 15:31:04.339180 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.339154 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4"] Jul 20 15:31:04.339329 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.339246 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.342611 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.342586 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 15:31:04.343158 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.343135 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-28jx6\"" Jul 20 15:31:04.343258 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.343186 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 15:31:04.442994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.442959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz7st\" (UniqueName: \"kubernetes.io/projected/6c3e8d87-817c-4fa4-bc0d-f790f11693f3-kube-api-access-dz7st\") pod \"openshift-lws-operator-fd8ccff4c-9psb4\" (UID: \"6c3e8d87-817c-4fa4-bc0d-f790f11693f3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.442994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.442997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c3e8d87-817c-4fa4-bc0d-f790f11693f3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9psb4\" (UID: \"6c3e8d87-817c-4fa4-bc0d-f790f11693f3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.543971 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.543887 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz7st\" (UniqueName: \"kubernetes.io/projected/6c3e8d87-817c-4fa4-bc0d-f790f11693f3-kube-api-access-dz7st\") pod \"openshift-lws-operator-fd8ccff4c-9psb4\" (UID: \"6c3e8d87-817c-4fa4-bc0d-f790f11693f3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.543971 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.543924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c3e8d87-817c-4fa4-bc0d-f790f11693f3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9psb4\" (UID: \"6c3e8d87-817c-4fa4-bc0d-f790f11693f3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.544250 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.544234 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c3e8d87-817c-4fa4-bc0d-f790f11693f3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9psb4\" (UID: \"6c3e8d87-817c-4fa4-bc0d-f790f11693f3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.554863 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.554837 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz7st\" (UniqueName: \"kubernetes.io/projected/6c3e8d87-817c-4fa4-bc0d-f790f11693f3-kube-api-access-dz7st\") pod \"openshift-lws-operator-fd8ccff4c-9psb4\" (UID: \"6c3e8d87-817c-4fa4-bc0d-f790f11693f3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.648116 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.648084 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" Jul 20 15:31:04.805736 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:04.805649 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4"] Jul 20 15:31:04.808661 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:04.808636 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c3e8d87_817c_4fa4_bc0d_f790f11693f3.slice/crio-73a9463eb0775d409aed9e0de088e1d02f5caea3b8bdf8b7a1f168e91575a8eb WatchSource:0}: Error finding container 73a9463eb0775d409aed9e0de088e1d02f5caea3b8bdf8b7a1f168e91575a8eb: Status 404 returned error can't find the container with id 73a9463eb0775d409aed9e0de088e1d02f5caea3b8bdf8b7a1f168e91575a8eb Jul 20 15:31:05.748321 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:05.748283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" event={"ID":"6c3e8d87-817c-4fa4-bc0d-f790f11693f3","Type":"ContainerStarted","Data":"73a9463eb0775d409aed9e0de088e1d02f5caea3b8bdf8b7a1f168e91575a8eb"} Jul 20 15:31:06.754348 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:06.754310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" event={"ID":"6c3e8d87-817c-4fa4-bc0d-f790f11693f3","Type":"ContainerStarted","Data":"07484069fb1a4cf65ab78a143ebd73fdebd6aea26d8cd6047e3619ad3406ef94"} Jul 20 15:31:06.779687 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:06.779613 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9psb4" podStartSLOduration=1.217977182 podStartE2EDuration="2.779598036s" totalImagesPullingTime="1.561620854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:06.777679662 +0000 UTC m=+258.722273046" watchObservedRunningTime="2026-07-20 15:31:06.779598036 +0000 UTC m=+258.724191406" Jul 20 15:31:09.956818 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:09.956718 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j"] Jul 20 15:31:09.959190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:09.959173 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:09.962188 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:09.962157 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7fjhx\"" Jul 20 15:31:09.962301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:09.962245 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 15:31:09.962974 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:09.962957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 15:31:09.971414 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:09.971392 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j"] Jul 20 15:31:10.089140 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.089100 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.089315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.089157 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gch8b\" (UniqueName: \"kubernetes.io/projected/c60f585d-5469-483c-8ed7-acbc32b868b5-kube-api-access-gch8b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.089315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.089201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.190421 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.190384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.190570 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.190448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gch8b\" (UniqueName: \"kubernetes.io/projected/c60f585d-5469-483c-8ed7-acbc32b868b5-kube-api-access-gch8b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.190570 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.190482 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.190791 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.190751 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.190887 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.190841 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.199287 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.199266 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gch8b\" (UniqueName: \"kubernetes.io/projected/c60f585d-5469-483c-8ed7-acbc32b868b5-kube-api-access-gch8b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.268681 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.268616 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:10.400969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.400952 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j"] Jul 20 15:31:10.403022 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:10.402995 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60f585d_5469_483c_8ed7_acbc32b868b5.slice/crio-7250f204c81fee4e3f06145d77002bbb0ac7bef37e9a14bbb6156e6bdde61986 WatchSource:0}: Error finding container 7250f204c81fee4e3f06145d77002bbb0ac7bef37e9a14bbb6156e6bdde61986: Status 404 returned error can't find the container with id 7250f204c81fee4e3f06145d77002bbb0ac7bef37e9a14bbb6156e6bdde61986 Jul 20 15:31:10.772917 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.772888 2557 generic.go:356] "Generic (PLEG): container finished" podID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerID="0653b3eb6e79215dcc4a288c380976b511112ad6763d593f75342f7e162a1750" exitCode=0 Jul 20 15:31:10.773066 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.772971 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" event={"ID":"c60f585d-5469-483c-8ed7-acbc32b868b5","Type":"ContainerDied","Data":"0653b3eb6e79215dcc4a288c380976b511112ad6763d593f75342f7e162a1750"} Jul 20 15:31:10.773066 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:10.773012 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" event={"ID":"c60f585d-5469-483c-8ed7-acbc32b868b5","Type":"ContainerStarted","Data":"7250f204c81fee4e3f06145d77002bbb0ac7bef37e9a14bbb6156e6bdde61986"} Jul 20 15:31:11.778841 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:11.778814 2557 generic.go:356] "Generic (PLEG): container finished" podID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerID="fb89998906e72bf531d8996b7344910659190cad203659f158db9294afb56697" exitCode=0 Jul 20 15:31:11.779193 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:11.778848 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" event={"ID":"c60f585d-5469-483c-8ed7-acbc32b868b5","Type":"ContainerDied","Data":"fb89998906e72bf531d8996b7344910659190cad203659f158db9294afb56697"} Jul 20 15:31:12.785749 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:12.785722 2557 generic.go:356] "Generic (PLEG): container finished" podID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerID="4e1ccedf3a121c6615bbcff20f8a9b7bdf45a7e63c8d18c750c2c047fbf689b8" exitCode=0 Jul 20 15:31:12.786124 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:12.785791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" event={"ID":"c60f585d-5469-483c-8ed7-acbc32b868b5","Type":"ContainerDied","Data":"4e1ccedf3a121c6615bbcff20f8a9b7bdf45a7e63c8d18c750c2c047fbf689b8"} Jul 20 15:31:13.914588 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:13.914565 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:14.022226 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.022192 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-bundle\") pod \"c60f585d-5469-483c-8ed7-acbc32b868b5\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " Jul 20 15:31:14.022397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.022284 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gch8b\" (UniqueName: \"kubernetes.io/projected/c60f585d-5469-483c-8ed7-acbc32b868b5-kube-api-access-gch8b\") pod \"c60f585d-5469-483c-8ed7-acbc32b868b5\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " Jul 20 15:31:14.022397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.022315 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-util\") pod \"c60f585d-5469-483c-8ed7-acbc32b868b5\" (UID: \"c60f585d-5469-483c-8ed7-acbc32b868b5\") " Jul 20 15:31:14.022925 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.022897 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-bundle" (OuterVolumeSpecName: "bundle") pod "c60f585d-5469-483c-8ed7-acbc32b868b5" (UID: "c60f585d-5469-483c-8ed7-acbc32b868b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:31:14.024459 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.024433 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60f585d-5469-483c-8ed7-acbc32b868b5-kube-api-access-gch8b" (OuterVolumeSpecName: "kube-api-access-gch8b") pod "c60f585d-5469-483c-8ed7-acbc32b868b5" (UID: "c60f585d-5469-483c-8ed7-acbc32b868b5"). InnerVolumeSpecName "kube-api-access-gch8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:31:14.030378 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.030336 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-util" (OuterVolumeSpecName: "util") pod "c60f585d-5469-483c-8ed7-acbc32b868b5" (UID: "c60f585d-5469-483c-8ed7-acbc32b868b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:31:14.123488 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.123452 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gch8b\" (UniqueName: \"kubernetes.io/projected/c60f585d-5469-483c-8ed7-acbc32b868b5-kube-api-access-gch8b\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:14.123488 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.123484 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:14.123667 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.123498 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60f585d-5469-483c-8ed7-acbc32b868b5-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:14.796217 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.796191 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" Jul 20 15:31:14.796397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.796228 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzm89j" event={"ID":"c60f585d-5469-483c-8ed7-acbc32b868b5","Type":"ContainerDied","Data":"7250f204c81fee4e3f06145d77002bbb0ac7bef37e9a14bbb6156e6bdde61986"} Jul 20 15:31:14.796397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:14.796274 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7250f204c81fee4e3f06145d77002bbb0ac7bef37e9a14bbb6156e6bdde61986" Jul 20 15:31:22.589542 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589514 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67"] Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589791 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="extract" Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589802 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="extract" Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589814 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="util" Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589819 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="util" Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589824 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="pull" Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589831 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="pull" Jul 20 15:31:22.589940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.589888 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c60f585d-5469-483c-8ed7-acbc32b868b5" containerName="extract" Jul 20 15:31:22.592323 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.592304 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.598534 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.598227 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 15:31:22.599077 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.599057 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 15:31:22.599174 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.599099 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 15:31:22.599290 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.599271 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 15:31:22.599564 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.599547 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k4m5s\"" Jul 20 15:31:22.615560 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.615531 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67"] Jul 20 15:31:22.691877 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.691852 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b4e6a08-1230-4abc-bf14-1d91f5f51040-manifests\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.691989 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.691895 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b4e6a08-1230-4abc-bf14-1d91f5f51040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.691989 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.691919 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9bzm\" (UniqueName: \"kubernetes.io/projected/7b4e6a08-1230-4abc-bf14-1d91f5f51040-kube-api-access-g9bzm\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.692124 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.691996 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b4e6a08-1230-4abc-bf14-1d91f5f51040-tmp\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.692124 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.692037 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b4e6a08-1230-4abc-bf14-1d91f5f51040-webhook-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.792590 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.792561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b4e6a08-1230-4abc-bf14-1d91f5f51040-tmp\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.792719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.792595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b4e6a08-1230-4abc-bf14-1d91f5f51040-webhook-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.792719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.792632 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b4e6a08-1230-4abc-bf14-1d91f5f51040-manifests\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.792719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.792681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b4e6a08-1230-4abc-bf14-1d91f5f51040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.792719 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.792716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9bzm\" (UniqueName: \"kubernetes.io/projected/7b4e6a08-1230-4abc-bf14-1d91f5f51040-kube-api-access-g9bzm\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.793122 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.793087 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b4e6a08-1230-4abc-bf14-1d91f5f51040-manifests\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.795013 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.794987 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b4e6a08-1230-4abc-bf14-1d91f5f51040-tmp\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.795317 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.795296 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b4e6a08-1230-4abc-bf14-1d91f5f51040-webhook-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.795357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.795296 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b4e6a08-1230-4abc-bf14-1d91f5f51040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.808519 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.808498 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9bzm\" (UniqueName: \"kubernetes.io/projected/7b4e6a08-1230-4abc-bf14-1d91f5f51040-kube-api-access-g9bzm\") pod \"opendatahub-operator-controller-manager-65bbbccfd9-d9f67\" (UID: \"7b4e6a08-1230-4abc-bf14-1d91f5f51040\") " pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:22.901549 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:22.901529 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:23.067073 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:23.067047 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67"] Jul 20 15:31:23.069577 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:23.069552 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4e6a08_1230_4abc_bf14_1d91f5f51040.slice/crio-b35162ec0a3af2a111ad3309987ebd9446574d0b7a9bfb8468a5c73c00e017d0 WatchSource:0}: Error finding container b35162ec0a3af2a111ad3309987ebd9446574d0b7a9bfb8468a5c73c00e017d0: Status 404 returned error can't find the container with id b35162ec0a3af2a111ad3309987ebd9446574d0b7a9bfb8468a5c73c00e017d0 Jul 20 15:31:23.839967 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:23.839929 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" event={"ID":"7b4e6a08-1230-4abc-bf14-1d91f5f51040","Type":"ContainerStarted","Data":"b35162ec0a3af2a111ad3309987ebd9446574d0b7a9bfb8468a5c73c00e017d0"} Jul 20 15:31:26.857760 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:26.857723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" event={"ID":"7b4e6a08-1230-4abc-bf14-1d91f5f51040","Type":"ContainerStarted","Data":"b596c7c5eed2a6ac65227302d8e03d7fad9f600ae1f3640ea23b82a3e96cf156"} Jul 20 15:31:27.868617 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:27.868566 2557 generic.go:356] "Generic (PLEG): container finished" podID="7b4e6a08-1230-4abc-bf14-1d91f5f51040" containerID="b596c7c5eed2a6ac65227302d8e03d7fad9f600ae1f3640ea23b82a3e96cf156" exitCode=0 Jul 20 15:31:27.869010 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:27.868630 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" event={"ID":"7b4e6a08-1230-4abc-bf14-1d91f5f51040","Type":"ContainerDied","Data":"b596c7c5eed2a6ac65227302d8e03d7fad9f600ae1f3640ea23b82a3e96cf156"} Jul 20 15:31:28.875440 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:28.875400 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" event={"ID":"7b4e6a08-1230-4abc-bf14-1d91f5f51040","Type":"ContainerStarted","Data":"692f2d3b37bdc3176e0ea3ef21d9aecef9572cbe7a25f353b92fb26cccbc9bba"} Jul 20 15:31:28.875923 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:28.875572 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:28.930026 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:28.929923 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" podStartSLOduration=1.972629021 podStartE2EDuration="6.929909087s" totalImagesPullingTime="3.953645143s" totalInitContainerRuntime="1.003634923s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:28.928213876 +0000 UTC m=+280.872807246" watchObservedRunningTime="2026-07-20 15:31:28.929909087 +0000 UTC m=+280.874502457" Jul 20 15:31:39.881439 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:39.881402 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65bbbccfd9-d9f67" Jul 20 15:31:42.962509 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.962476 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s"] Jul 20 15:31:42.964906 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.964890 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:42.974005 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.973987 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 15:31:42.975122 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.975108 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 15:31:42.977492 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.977475 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 15:31:42.977994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.977976 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cd8kp\"" Jul 20 15:31:42.978113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:42.977980 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 15:31:43.042647 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.042609 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s"] Jul 20 15:31:43.160158 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.160122 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zh5m\" (UniqueName: \"kubernetes.io/projected/a557493c-e288-4800-9557-e5e63567342e-kube-api-access-8zh5m\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.160309 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.160175 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a557493c-e288-4800-9557-e5e63567342e-tmp\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.160309 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.160199 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a557493c-e288-4800-9557-e5e63567342e-tls-certs\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.260984 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.260892 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a557493c-e288-4800-9557-e5e63567342e-tmp\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.260984 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.260932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a557493c-e288-4800-9557-e5e63567342e-tls-certs\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.260984 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.260985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zh5m\" (UniqueName: \"kubernetes.io/projected/a557493c-e288-4800-9557-e5e63567342e-kube-api-access-8zh5m\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.263330 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.263299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a557493c-e288-4800-9557-e5e63567342e-tmp\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.263516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.263497 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a557493c-e288-4800-9557-e5e63567342e-tls-certs\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.279889 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.279860 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zh5m\" (UniqueName: \"kubernetes.io/projected/a557493c-e288-4800-9557-e5e63567342e-kube-api-access-8zh5m\") pod \"kube-auth-proxy-6598cc5cf9-ltf4s\" (UID: \"a557493c-e288-4800-9557-e5e63567342e\") " pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.574328 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.574246 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" Jul 20 15:31:43.723422 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.723389 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s"] Jul 20 15:31:43.726684 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:43.726660 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda557493c_e288_4800_9557_e5e63567342e.slice/crio-0bbe743ec6a55e774388b8213f2d15c06e8deed4ce6ec4c3d1d5485b0c8ffb7e WatchSource:0}: Error finding container 0bbe743ec6a55e774388b8213f2d15c06e8deed4ce6ec4c3d1d5485b0c8ffb7e: Status 404 returned error can't find the container with id 0bbe743ec6a55e774388b8213f2d15c06e8deed4ce6ec4c3d1d5485b0c8ffb7e Jul 20 15:31:43.945855 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:43.945824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" event={"ID":"a557493c-e288-4800-9557-e5e63567342e","Type":"ContainerStarted","Data":"0bbe743ec6a55e774388b8213f2d15c06e8deed4ce6ec4c3d1d5485b0c8ffb7e"} Jul 20 15:31:44.209516 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.209439 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zsftq"] Jul 20 15:31:44.212373 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.212352 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.216089 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.216063 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 20 15:31:44.219152 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.219070 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qzdvx\"" Jul 20 15:31:44.258971 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.258943 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zsftq"] Jul 20 15:31:44.371167 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.371128 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36976c57-7182-4cac-bba4-7bbd44759099-tls-certs\") pod \"model-serving-api-54d89bd5fc-zsftq\" (UID: \"36976c57-7182-4cac-bba4-7bbd44759099\") " pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.371357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.371183 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5dk9\" (UniqueName: \"kubernetes.io/projected/36976c57-7182-4cac-bba4-7bbd44759099-kube-api-access-j5dk9\") pod \"model-serving-api-54d89bd5fc-zsftq\" (UID: \"36976c57-7182-4cac-bba4-7bbd44759099\") " pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.472022 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.471941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36976c57-7182-4cac-bba4-7bbd44759099-tls-certs\") pod \"model-serving-api-54d89bd5fc-zsftq\" (UID: \"36976c57-7182-4cac-bba4-7bbd44759099\") " pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.472022 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.471987 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5dk9\" (UniqueName: \"kubernetes.io/projected/36976c57-7182-4cac-bba4-7bbd44759099-kube-api-access-j5dk9\") pod \"model-serving-api-54d89bd5fc-zsftq\" (UID: \"36976c57-7182-4cac-bba4-7bbd44759099\") " pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.475121 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.475095 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36976c57-7182-4cac-bba4-7bbd44759099-tls-certs\") pod \"model-serving-api-54d89bd5fc-zsftq\" (UID: \"36976c57-7182-4cac-bba4-7bbd44759099\") " pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.486911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.486859 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5dk9\" (UniqueName: \"kubernetes.io/projected/36976c57-7182-4cac-bba4-7bbd44759099-kube-api-access-j5dk9\") pod \"model-serving-api-54d89bd5fc-zsftq\" (UID: \"36976c57-7182-4cac-bba4-7bbd44759099\") " pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.522560 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.522533 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:44.687579 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.687550 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zsftq"] Jul 20 15:31:44.690053 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:44.690024 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36976c57_7182_4cac_bba4_7bbd44759099.slice/crio-335108b6bf0c617739d930b2208554bf194f45fcdaf7b8f06eca40ed04ed548e WatchSource:0}: Error finding container 335108b6bf0c617739d930b2208554bf194f45fcdaf7b8f06eca40ed04ed548e: Status 404 returned error can't find the container with id 335108b6bf0c617739d930b2208554bf194f45fcdaf7b8f06eca40ed04ed548e Jul 20 15:31:44.952723 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:44.952683 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" event={"ID":"36976c57-7182-4cac-bba4-7bbd44759099","Type":"ContainerStarted","Data":"335108b6bf0c617739d930b2208554bf194f45fcdaf7b8f06eca40ed04ed548e"} Jul 20 15:31:45.174167 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.174132 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n62hz"] Jul 20 15:31:45.177291 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.177268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.179470 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.179434 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 20 15:31:45.180113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.180092 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 20 15:31:45.180258 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.180128 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 15:31:45.180258 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.180184 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2qnhd\"" Jul 20 15:31:45.180471 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.180451 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 20 15:31:45.207680 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.207616 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n62hz"] Jul 20 15:31:45.277235 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277204 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.277623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277257 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-istio-token\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.277623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277285 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-cacerts\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.277623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.277623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04576a6-87d4-4637-b136-c01275b6a972-local-certs\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.277623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277454 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.277623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.277483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d9nm\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-kube-api-access-8d9nm\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378512 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378281 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378512 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04576a6-87d4-4637-b136-c01275b6a972-local-certs\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378512 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378512 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d9nm\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-kube-api-access-8d9nm\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378512 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378504 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378898 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-istio-token\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.378898 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.378566 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-cacerts\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.379722 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.379653 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.381505 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.381479 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.381596 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.381524 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04576a6-87d4-4637-b136-c01275b6a972-local-certs\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.382085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.382062 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.383116 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.383086 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-cacerts\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.389986 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.389962 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-istio-token\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.390944 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.390920 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d9nm\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-kube-api-access-8d9nm\") pod \"istiod-openshift-gateway-7574459656-n62hz\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.488793 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.488705 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:45.669636 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:45.669603 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n62hz"] Jul 20 15:31:46.316802 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:46.316752 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04576a6_87d4_4637_b136_c01275b6a972.slice/crio-1792ae3325b94cddb731f3ae17904b561fe38e8ad75afb72b1ab39d9964e1df4 WatchSource:0}: Error finding container 1792ae3325b94cddb731f3ae17904b561fe38e8ad75afb72b1ab39d9964e1df4: Status 404 returned error can't find the container with id 1792ae3325b94cddb731f3ae17904b561fe38e8ad75afb72b1ab39d9964e1df4 Jul 20 15:31:46.966331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:46.966042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" event={"ID":"b04576a6-87d4-4637-b136-c01275b6a972","Type":"ContainerStarted","Data":"1792ae3325b94cddb731f3ae17904b561fe38e8ad75afb72b1ab39d9964e1df4"} Jul 20 15:31:46.969394 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:46.969328 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" event={"ID":"a557493c-e288-4800-9557-e5e63567342e","Type":"ContainerStarted","Data":"a35c54413a5e67912a04f3966f2a73da2bbee2e926ae4a36154b6823b549ad23"} Jul 20 15:31:46.971481 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:46.971452 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" event={"ID":"36976c57-7182-4cac-bba4-7bbd44759099","Type":"ContainerStarted","Data":"8be15bf0233039aa8bdc1c2b36a94a6442dd40ec992284cb471eb3d761d7f821"} Jul 20 15:31:46.971627 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:46.971610 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:46.998930 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:46.998870 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6598cc5cf9-ltf4s" podStartSLOduration=2.020262298 podStartE2EDuration="4.998854694s" totalImagesPullingTime="2.978592396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:46.996142077 +0000 UTC m=+298.940735446" watchObservedRunningTime="2026-07-20 15:31:46.998854694 +0000 UTC m=+298.943448063" Jul 20 15:31:47.021371 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:47.021290 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" podStartSLOduration=0.987541823 podStartE2EDuration="3.021272724s" totalImagesPullingTime="2.033730901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:47.019530067 +0000 UTC m=+298.964123440" watchObservedRunningTime="2026-07-20 15:31:47.021272724 +0000 UTC m=+298.965866094" Jul 20 15:31:48.636386 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.636354 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:31:48.637685 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.636834 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:31:48.641595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.641576 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 15:31:48.652541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.652508 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:31:48.652592 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.652574 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:31:48.984217 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.984139 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" event={"ID":"b04576a6-87d4-4637-b136-c01275b6a972","Type":"ContainerStarted","Data":"2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f"} Jul 20 15:31:48.984217 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:48.984196 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:49.013178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:49.013123 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" podStartSLOduration=1.680121048 podStartE2EDuration="4.013103979s" totalImagesPullingTime="2.332982931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:49.012035112 +0000 UTC m=+300.956628511" watchObservedRunningTime="2026-07-20 15:31:49.013103979 +0000 UTC m=+300.957697340" Jul 20 15:31:49.990122 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:49.990084 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-n62hz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 15:31:49.990496 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:49.990150 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" podUID="b04576a6-87d4-4637-b136-c01275b6a972" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 15:31:50.533963 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.533937 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv"] Jul 20 15:31:50.537442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.537411 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.540983 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.540963 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4pzqc\"" Jul 20 15:31:50.555977 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.555955 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv"] Jul 20 15:31:50.627701 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627671 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627826 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627714 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627826 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627826 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627762 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627826 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627807 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a8ea7-a880-4ecb-bf88-1b36beef776e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627963 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627963 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627909 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627963 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627932 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.627963 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.627951 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx6f8\" (UniqueName: \"kubernetes.io/projected/403a8ea7-a880-4ecb-bf88-1b36beef776e-kube-api-access-dx6f8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729202 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729202 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729226 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a8ea7-a880-4ecb-bf88-1b36beef776e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729389 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729345 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729422 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx6f8\" (UniqueName: \"kubernetes.io/projected/403a8ea7-a880-4ecb-bf88-1b36beef776e-kube-api-access-dx6f8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729633 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729633 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729633 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729597 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729819 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729733 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.729908 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729890 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.730122 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.729944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.730229 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.730207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403a8ea7-a880-4ecb-bf88-1b36beef776e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.731672 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.731636 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.731971 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.731954 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.739842 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.739750 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403a8ea7-a880-4ecb-bf88-1b36beef776e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.744047 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.744030 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx6f8\" (UniqueName: \"kubernetes.io/projected/403a8ea7-a880-4ecb-bf88-1b36beef776e-kube-api-access-dx6f8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gm7pv\" (UID: \"403a8ea7-a880-4ecb-bf88-1b36beef776e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.848051 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.847979 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:50.989466 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.989434 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv"] Jul 20 15:31:50.992819 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:50.992791 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403a8ea7_a880_4ecb_bf88_1b36beef776e.slice/crio-00802698279f0e92eeff2eae5f82475bc476fc584ab1cf302d1e34d804fc4b2d WatchSource:0}: Error finding container 00802698279f0e92eeff2eae5f82475bc476fc584ab1cf302d1e34d804fc4b2d: Status 404 returned error can't find the container with id 00802698279f0e92eeff2eae5f82475bc476fc584ab1cf302d1e34d804fc4b2d Jul 20 15:31:50.994662 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:50.994646 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 15:31:52.000710 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:52.000665 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" event={"ID":"403a8ea7-a880-4ecb-bf88-1b36beef776e","Type":"ContainerStarted","Data":"00802698279f0e92eeff2eae5f82475bc476fc584ab1cf302d1e34d804fc4b2d"} Jul 20 15:31:52.989700 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:52.989668 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:53.866936 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:53.866904 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:31:53.867182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:53.866974 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:31:53.867182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:53.867004 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:31:54.014387 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:54.014346 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" event={"ID":"403a8ea7-a880-4ecb-bf88-1b36beef776e","Type":"ContainerStarted","Data":"2bf11c5e6804c4be5968a50615d715303e8fedc844061dad5fd0c72339706f9c"} Jul 20 15:31:54.065836 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:54.065753 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" podStartSLOduration=1.1939104170000001 podStartE2EDuration="4.065737438s" totalImagesPullingTime="2.871827021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:54.062417445 +0000 UTC m=+306.007010815" watchObservedRunningTime="2026-07-20 15:31:54.065737438 +0000 UTC m=+306.010330878" Jul 20 15:31:54.848674 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:54.848637 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:54.853326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:54.853302 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:55.026058 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.026022 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:55.027187 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.027166 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gm7pv" Jul 20 15:31:55.192064 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.192033 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dxglr"] Jul 20 15:31:55.195346 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.195322 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.197321 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.197297 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 20 15:31:55.197429 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.197404 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z4rvw\"" Jul 20 15:31:55.208430 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.208409 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dxglr"] Jul 20 15:31:55.272130 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.272095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc609f54-fe6f-49d6-8239-85efc7351571-cert\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.272278 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.272132 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdt48\" (UniqueName: \"kubernetes.io/projected/cc609f54-fe6f-49d6-8239-85efc7351571-kube-api-access-pdt48\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.372823 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.372791 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc609f54-fe6f-49d6-8239-85efc7351571-cert\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.372959 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.372833 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdt48\" (UniqueName: \"kubernetes.io/projected/cc609f54-fe6f-49d6-8239-85efc7351571-kube-api-access-pdt48\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.372959 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:31:55.372925 2557 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 20 15:31:55.373049 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:31:55.372989 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc609f54-fe6f-49d6-8239-85efc7351571-cert podName:cc609f54-fe6f-49d6-8239-85efc7351571 nodeName:}" failed. No retries permitted until 2026-07-20 15:31:55.8729717 +0000 UTC m=+307.817565051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc609f54-fe6f-49d6-8239-85efc7351571-cert") pod "kserve-controller-manager-d6f4b65b8-dxglr" (UID: "cc609f54-fe6f-49d6-8239-85efc7351571") : secret "kserve-webhook-server-cert" not found Jul 20 15:31:55.390018 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.389990 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdt48\" (UniqueName: \"kubernetes.io/projected/cc609f54-fe6f-49d6-8239-85efc7351571-kube-api-access-pdt48\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.876611 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.876585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc609f54-fe6f-49d6-8239-85efc7351571-cert\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:55.879033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:55.879014 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc609f54-fe6f-49d6-8239-85efc7351571-cert\") pod \"kserve-controller-manager-d6f4b65b8-dxglr\" (UID: \"cc609f54-fe6f-49d6-8239-85efc7351571\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:56.106292 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:56.106259 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:56.270207 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:31:56.270178 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc609f54_fe6f_49d6_8239_85efc7351571.slice/crio-5deac100bd98fad08ae65d946f7a31083b5da94053d106eec2cc231510fc7bc5 WatchSource:0}: Error finding container 5deac100bd98fad08ae65d946f7a31083b5da94053d106eec2cc231510fc7bc5: Status 404 returned error can't find the container with id 5deac100bd98fad08ae65d946f7a31083b5da94053d106eec2cc231510fc7bc5 Jul 20 15:31:56.270485 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:56.270460 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dxglr"] Jul 20 15:31:57.037486 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:57.037449 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" event={"ID":"cc609f54-fe6f-49d6-8239-85efc7351571","Type":"ContainerStarted","Data":"5deac100bd98fad08ae65d946f7a31083b5da94053d106eec2cc231510fc7bc5"} Jul 20 15:31:57.981486 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:57.981457 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zsftq" Jul 20 15:31:58.320702 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:58.320611 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n62hz"] Jul 20 15:31:58.320941 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:58.320846 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" podUID="b04576a6-87d4-4637-b136-c01275b6a972" containerName="discovery" containerID="cri-o://2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f" gracePeriod=30 Jul 20 15:31:59.015378 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.015357 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:59.050716 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.050675 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" event={"ID":"cc609f54-fe6f-49d6-8239-85efc7351571","Type":"ContainerStarted","Data":"2ea2cc997704ed251de979110f38b54782923802314f05ae7f451705a33cc70b"} Jul 20 15:31:59.050930 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.050911 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:31:59.052239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.052216 2557 generic.go:356] "Generic (PLEG): container finished" podID="b04576a6-87d4-4637-b136-c01275b6a972" containerID="2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f" exitCode=0 Jul 20 15:31:59.052357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.052248 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" event={"ID":"b04576a6-87d4-4637-b136-c01275b6a972","Type":"ContainerDied","Data":"2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f"} Jul 20 15:31:59.052357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.052276 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" Jul 20 15:31:59.052357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.052284 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" event={"ID":"b04576a6-87d4-4637-b136-c01275b6a972","Type":"ContainerDied","Data":"1792ae3325b94cddb731f3ae17904b561fe38e8ad75afb72b1ab39d9964e1df4"} Jul 20 15:31:59.052357 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.052312 2557 scope.go:117] "RemoveContainer" containerID="2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f" Jul 20 15:31:59.061223 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.061200 2557 scope.go:117] "RemoveContainer" containerID="2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f" Jul 20 15:31:59.061566 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:31:59.061517 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f\": container with ID starting with 2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f not found: ID does not exist" containerID="2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f" Jul 20 15:31:59.061678 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.061575 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f"} err="failed to get container status \"2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f\": rpc error: code = NotFound desc = could not find container \"2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f\": container with ID starting with 2a37c74f2f7a219eeedea97274892b40aa868823faf24558a0ba39a1cb80661f not found: ID does not exist" Jul 20 15:31:59.072831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.072746 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" podStartSLOduration=1.43415209 podStartE2EDuration="4.072727355s" totalImagesPullingTime="2.638575265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:31:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:31:59.070566217 +0000 UTC m=+311.015159584" watchObservedRunningTime="2026-07-20 15:31:59.072727355 +0000 UTC m=+311.017320726" Jul 20 15:31:59.105941 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.105915 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-kubeconfig\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106054 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.105970 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-cacerts\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106054 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.106008 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04576a6-87d4-4637-b136-c01275b6a972-local-certs\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106054 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.106028 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-ca-configmap\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.106066 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-dns-cert\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.106093 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d9nm\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-kube-api-access-8d9nm\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106239 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.106127 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-istio-token\") pod \"b04576a6-87d4-4637-b136-c01275b6a972\" (UID: \"b04576a6-87d4-4637-b136-c01275b6a972\") " Jul 20 15:31:59.106655 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.106391 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:31:59.108747 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.108716 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b04576a6-87d4-4637-b136-c01275b6a972-local-certs" (OuterVolumeSpecName: "local-certs") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:31:59.108879 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.108843 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:31:59.108966 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.108885 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:31:59.108966 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.108914 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-cacerts" (OuterVolumeSpecName: "cacerts") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:31:59.109078 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.109008 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-istio-token" (OuterVolumeSpecName: "istio-token") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:31:59.109246 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.109226 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-kube-api-access-8d9nm" (OuterVolumeSpecName: "kube-api-access-8d9nm") pod "b04576a6-87d4-4637-b136-c01275b6a972" (UID: "b04576a6-87d4-4637-b136-c01275b6a972"). InnerVolumeSpecName "kube-api-access-8d9nm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:31:59.207924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207845 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d9nm\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-kube-api-access-8d9nm\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.207924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207879 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b04576a6-87d4-4637-b136-c01275b6a972-istio-token\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.207924 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207917 2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-kubeconfig\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.208125 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207930 2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-cacerts\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.208125 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207943 2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b04576a6-87d4-4637-b136-c01275b6a972-local-certs\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.208125 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207955 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-ca-configmap\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.208125 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.207966 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b04576a6-87d4-4637-b136-c01275b6a972-istio-csr-dns-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:31:59.382825 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.382796 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n62hz"] Jul 20 15:31:59.386007 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:31:59.385986 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n62hz"] Jul 20 15:32:00.565970 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:00.565936 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b04576a6-87d4-4637-b136-c01275b6a972" path="/var/lib/kubelet/pods/b04576a6-87d4-4637-b136-c01275b6a972/volumes" Jul 20 15:32:03.989188 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:03.989150 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-n62hz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 15:32:03.989574 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:03.989208 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n62hz" podUID="b04576a6-87d4-4637-b136-c01275b6a972" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.30:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 15:32:30.064064 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:30.064036 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dxglr" Jul 20 15:32:43.560447 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.560412 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4m56"] Jul 20 15:32:43.560892 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.560760 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b04576a6-87d4-4637-b136-c01275b6a972" containerName="discovery" Jul 20 15:32:43.560892 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.560798 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b04576a6-87d4-4637-b136-c01275b6a972" containerName="discovery" Jul 20 15:32:43.560892 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.560875 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b04576a6-87d4-4637-b136-c01275b6a972" containerName="discovery" Jul 20 15:32:43.563519 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.563501 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:43.565665 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.565633 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-f54fb\"" Jul 20 15:32:43.565759 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.565633 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 15:32:43.566020 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.566001 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 15:32:43.572473 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.572450 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4m56"] Jul 20 15:32:43.666131 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.666091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g72fh\" (UniqueName: \"kubernetes.io/projected/53b4d44c-8f27-41a7-b4a5-9071bf66001e-kube-api-access-g72fh\") pod \"kuadrant-operator-catalog-c4m56\" (UID: \"53b4d44c-8f27-41a7-b4a5-9071bf66001e\") " pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:43.767530 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.767490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g72fh\" (UniqueName: \"kubernetes.io/projected/53b4d44c-8f27-41a7-b4a5-9071bf66001e-kube-api-access-g72fh\") pod \"kuadrant-operator-catalog-c4m56\" (UID: \"53b4d44c-8f27-41a7-b4a5-9071bf66001e\") " pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:43.775807 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.775763 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g72fh\" (UniqueName: \"kubernetes.io/projected/53b4d44c-8f27-41a7-b4a5-9071bf66001e-kube-api-access-g72fh\") pod \"kuadrant-operator-catalog-c4m56\" (UID: \"53b4d44c-8f27-41a7-b4a5-9071bf66001e\") " pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:43.873688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.873661 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:43.927495 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:43.927465 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4m56"] Jul 20 15:32:44.002630 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.002604 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4m56"] Jul 20 15:32:44.005603 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:32:44.005575 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b4d44c_8f27_41a7_b4a5_9071bf66001e.slice/crio-b33f3be538d930f238cd7d58faefedd2d7a59ad5efeb41c9920b73e97d12425a WatchSource:0}: Error finding container b33f3be538d930f238cd7d58faefedd2d7a59ad5efeb41c9920b73e97d12425a: Status 404 returned error can't find the container with id b33f3be538d930f238cd7d58faefedd2d7a59ad5efeb41c9920b73e97d12425a Jul 20 15:32:44.135507 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.135422 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f4nwv"] Jul 20 15:32:44.141537 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.141509 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:44.147073 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.147045 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f4nwv"] Jul 20 15:32:44.171406 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.171383 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvgql\" (UniqueName: \"kubernetes.io/projected/91ac4863-7768-4482-843d-f8a2f3b76f75-kube-api-access-gvgql\") pod \"kuadrant-operator-catalog-f4nwv\" (UID: \"91ac4863-7768-4482-843d-f8a2f3b76f75\") " pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:44.263592 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.263559 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" event={"ID":"53b4d44c-8f27-41a7-b4a5-9071bf66001e","Type":"ContainerStarted","Data":"b33f3be538d930f238cd7d58faefedd2d7a59ad5efeb41c9920b73e97d12425a"} Jul 20 15:32:44.271996 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.271972 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvgql\" (UniqueName: \"kubernetes.io/projected/91ac4863-7768-4482-843d-f8a2f3b76f75-kube-api-access-gvgql\") pod \"kuadrant-operator-catalog-f4nwv\" (UID: \"91ac4863-7768-4482-843d-f8a2f3b76f75\") " pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:44.280994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.280965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvgql\" (UniqueName: \"kubernetes.io/projected/91ac4863-7768-4482-843d-f8a2f3b76f75-kube-api-access-gvgql\") pod \"kuadrant-operator-catalog-f4nwv\" (UID: \"91ac4863-7768-4482-843d-f8a2f3b76f75\") " pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:44.452265 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.452177 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:44.580697 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:44.580653 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f4nwv"] Jul 20 15:32:44.583296 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:32:44.583270 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ac4863_7768_4482_843d_f8a2f3b76f75.slice/crio-a31091ab15e0fac4e2891180f8281181abeb25fdeb55cc87330dd692cda5f8ab WatchSource:0}: Error finding container a31091ab15e0fac4e2891180f8281181abeb25fdeb55cc87330dd692cda5f8ab: Status 404 returned error can't find the container with id a31091ab15e0fac4e2891180f8281181abeb25fdeb55cc87330dd692cda5f8ab Jul 20 15:32:45.270814 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:45.270739 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" event={"ID":"91ac4863-7768-4482-843d-f8a2f3b76f75","Type":"ContainerStarted","Data":"a31091ab15e0fac4e2891180f8281181abeb25fdeb55cc87330dd692cda5f8ab"} Jul 20 15:32:47.281451 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.281413 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" event={"ID":"53b4d44c-8f27-41a7-b4a5-9071bf66001e","Type":"ContainerStarted","Data":"70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8"} Jul 20 15:32:47.281938 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.281445 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" podUID="53b4d44c-8f27-41a7-b4a5-9071bf66001e" containerName="registry-server" containerID="cri-o://70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8" gracePeriod=2 Jul 20 15:32:47.283149 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.283114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" event={"ID":"91ac4863-7768-4482-843d-f8a2f3b76f75","Type":"ContainerStarted","Data":"f9bda501ea0e02fae613f3ef08ca0561b8f97b99c0c0a61c84516e114cb730ae"} Jul 20 15:32:47.297687 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.297645 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" podStartSLOduration=2.007889141 podStartE2EDuration="4.297631989s" totalImagesPullingTime="2.289742848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:32:47.295904948 +0000 UTC m=+359.240498318" watchObservedRunningTime="2026-07-20 15:32:47.297631989 +0000 UTC m=+359.242225358" Jul 20 15:32:47.311577 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.311531 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" podStartSLOduration=1.596155425 podStartE2EDuration="3.311516496s" totalImagesPullingTime="1.715361071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:32:47.310762181 +0000 UTC m=+359.255355550" watchObservedRunningTime="2026-07-20 15:32:47.311516496 +0000 UTC m=+359.256109866" Jul 20 15:32:47.521435 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.521409 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:47.604156 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.604093 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g72fh\" (UniqueName: \"kubernetes.io/projected/53b4d44c-8f27-41a7-b4a5-9071bf66001e-kube-api-access-g72fh\") pod \"53b4d44c-8f27-41a7-b4a5-9071bf66001e\" (UID: \"53b4d44c-8f27-41a7-b4a5-9071bf66001e\") " Jul 20 15:32:47.606225 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.606203 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b4d44c-8f27-41a7-b4a5-9071bf66001e-kube-api-access-g72fh" (OuterVolumeSpecName: "kube-api-access-g72fh") pod "53b4d44c-8f27-41a7-b4a5-9071bf66001e" (UID: "53b4d44c-8f27-41a7-b4a5-9071bf66001e"). InnerVolumeSpecName "kube-api-access-g72fh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:32:47.704576 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:47.704545 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g72fh\" (UniqueName: \"kubernetes.io/projected/53b4d44c-8f27-41a7-b4a5-9071bf66001e-kube-api-access-g72fh\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:32:48.289623 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.289591 2557 generic.go:356] "Generic (PLEG): container finished" podID="53b4d44c-8f27-41a7-b4a5-9071bf66001e" containerID="70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8" exitCode=0 Jul 20 15:32:48.290113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.289636 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" Jul 20 15:32:48.290113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.289635 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" event={"ID":"53b4d44c-8f27-41a7-b4a5-9071bf66001e","Type":"ContainerDied","Data":"70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8"} Jul 20 15:32:48.290113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.289694 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c4m56" event={"ID":"53b4d44c-8f27-41a7-b4a5-9071bf66001e","Type":"ContainerDied","Data":"b33f3be538d930f238cd7d58faefedd2d7a59ad5efeb41c9920b73e97d12425a"} Jul 20 15:32:48.290113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.289713 2557 scope.go:117] "RemoveContainer" containerID="70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8" Jul 20 15:32:48.298978 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.298962 2557 scope.go:117] "RemoveContainer" containerID="70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8" Jul 20 15:32:48.299221 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:32:48.299201 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8\": container with ID starting with 70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8 not found: ID does not exist" containerID="70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8" Jul 20 15:32:48.299282 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.299230 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8"} err="failed to get container status \"70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8\": rpc error: code = NotFound desc = could not find container \"70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8\": container with ID starting with 70a63702823dd5920fa18fd6a0dbda6e49e544e7a4c5becbb03b1c404c705ce8 not found: ID does not exist" Jul 20 15:32:48.309520 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.309495 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4m56"] Jul 20 15:32:48.314165 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.314146 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4m56"] Jul 20 15:32:48.566679 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:48.566587 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53b4d44c-8f27-41a7-b4a5-9071bf66001e" path="/var/lib/kubelet/pods/53b4d44c-8f27-41a7-b4a5-9071bf66001e/volumes" Jul 20 15:32:54.452607 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:54.452574 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:54.453026 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:54.452665 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:54.475186 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:54.475163 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:55.343143 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:55.343118 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-f4nwv" Jul 20 15:32:59.557744 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.557710 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l"] Jul 20 15:32:59.558115 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.558047 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b4d44c-8f27-41a7-b4a5-9071bf66001e" containerName="registry-server" Jul 20 15:32:59.558115 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.558059 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b4d44c-8f27-41a7-b4a5-9071bf66001e" containerName="registry-server" Jul 20 15:32:59.558187 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.558123 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b4d44c-8f27-41a7-b4a5-9071bf66001e" containerName="registry-server" Jul 20 15:32:59.562352 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.562336 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.564434 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.564413 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x6bxm\"" Jul 20 15:32:59.570983 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.570958 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l"] Jul 20 15:32:59.698290 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.698259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.698290 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.698292 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.698466 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.698351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ffp\" (UniqueName: \"kubernetes.io/projected/be2760a3-1789-42bc-9f82-41044e67d6e7-kube-api-access-r9ffp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.799676 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.799630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.799885 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.799681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.799885 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.799754 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ffp\" (UniqueName: \"kubernetes.io/projected/be2760a3-1789-42bc-9f82-41044e67d6e7-kube-api-access-r9ffp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.800102 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.800078 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.800160 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.800109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.807154 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.807137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ffp\" (UniqueName: \"kubernetes.io/projected/be2760a3-1789-42bc-9f82-41044e67d6e7-kube-api-access-r9ffp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:32:59.871961 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:32:59.871883 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:33:00.007978 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.007949 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l"] Jul 20 15:33:00.010490 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:00.010466 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2760a3_1789_42bc_9f82_41044e67d6e7.slice/crio-cef664a5a247bf7a1a11966bbf2fc187a8a7791fe4ecaf9fc6de78d5516ddc52 WatchSource:0}: Error finding container cef664a5a247bf7a1a11966bbf2fc187a8a7791fe4ecaf9fc6de78d5516ddc52: Status 404 returned error can't find the container with id cef664a5a247bf7a1a11966bbf2fc187a8a7791fe4ecaf9fc6de78d5516ddc52 Jul 20 15:33:00.155068 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.155040 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5"] Jul 20 15:33:00.158561 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.158545 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.165666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.165643 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5"] Jul 20 15:33:00.303465 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.303436 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr8q2\" (UniqueName: \"kubernetes.io/projected/df101e46-4c7d-4942-864a-4587988ed46d-kube-api-access-wr8q2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.303587 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.303498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.303587 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.303535 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.346395 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.346370 2557 generic.go:356] "Generic (PLEG): container finished" podID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerID="9893dd571a709f867d7fdbdedfe411dc1f49f696a1681b98def462d8d88c2c2a" exitCode=0 Jul 20 15:33:00.346490 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.346440 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" event={"ID":"be2760a3-1789-42bc-9f82-41044e67d6e7","Type":"ContainerDied","Data":"9893dd571a709f867d7fdbdedfe411dc1f49f696a1681b98def462d8d88c2c2a"} Jul 20 15:33:00.346534 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.346490 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" event={"ID":"be2760a3-1789-42bc-9f82-41044e67d6e7","Type":"ContainerStarted","Data":"cef664a5a247bf7a1a11966bbf2fc187a8a7791fe4ecaf9fc6de78d5516ddc52"} Jul 20 15:33:00.404557 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.404529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr8q2\" (UniqueName: \"kubernetes.io/projected/df101e46-4c7d-4942-864a-4587988ed46d-kube-api-access-wr8q2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.404684 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.404666 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.404729 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.404709 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.405046 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.405031 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.405109 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.405075 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.412265 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.412247 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr8q2\" (UniqueName: \"kubernetes.io/projected/df101e46-4c7d-4942-864a-4587988ed46d-kube-api-access-wr8q2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.480692 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.480667 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:00.555974 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.555945 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz"] Jul 20 15:33:00.568570 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.566086 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.587529 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.587501 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz"] Jul 20 15:33:00.620589 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.620567 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5"] Jul 20 15:33:00.622109 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:00.622082 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf101e46_4c7d_4942_864a_4587988ed46d.slice/crio-4be64c80f160184a2561bacabb02c3d9d0c9f514c5b7914f4e999e22635df2d4 WatchSource:0}: Error finding container 4be64c80f160184a2561bacabb02c3d9d0c9f514c5b7914f4e999e22635df2d4: Status 404 returned error can't find the container with id 4be64c80f160184a2561bacabb02c3d9d0c9f514c5b7914f4e999e22635df2d4 Jul 20 15:33:00.707545 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.707513 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4cst\" (UniqueName: \"kubernetes.io/projected/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-kube-api-access-s4cst\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.707701 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.707551 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.707701 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.707583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.808519 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.808493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.808637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.808567 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4cst\" (UniqueName: \"kubernetes.io/projected/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-kube-api-access-s4cst\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.808637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.808601 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.808939 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.808919 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.808977 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.808944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.816987 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.816965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4cst\" (UniqueName: \"kubernetes.io/projected/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-kube-api-access-s4cst\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.880260 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.880238 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:00.951362 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.951333 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn"] Jul 20 15:33:00.955176 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.955157 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:00.967457 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:00.967428 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn"] Jul 20 15:33:01.018173 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.018151 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz"] Jul 20 15:33:01.037931 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:01.037901 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cfb23c_0d5d_4ffe_bb23_4d81dc85213e.slice/crio-0176691fefa2ebf46a8e759cfbcd1a177306de96a70cd37d9e594d9953f5ee3e WatchSource:0}: Error finding container 0176691fefa2ebf46a8e759cfbcd1a177306de96a70cd37d9e594d9953f5ee3e: Status 404 returned error can't find the container with id 0176691fefa2ebf46a8e759cfbcd1a177306de96a70cd37d9e594d9953f5ee3e Jul 20 15:33:01.112518 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.112493 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.112620 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.112552 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvvm9\" (UniqueName: \"kubernetes.io/projected/bbadba61-e355-4b28-9725-a9b76ff809f7-kube-api-access-zvvm9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.112683 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.112626 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.213955 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.213927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.214039 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.213966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.214039 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.214011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvvm9\" (UniqueName: \"kubernetes.io/projected/bbadba61-e355-4b28-9725-a9b76ff809f7-kube-api-access-zvvm9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.214364 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.214342 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.214406 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.214347 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.222143 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.222123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvvm9\" (UniqueName: \"kubernetes.io/projected/bbadba61-e355-4b28-9725-a9b76ff809f7-kube-api-access-zvvm9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.267033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.267007 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:01.354926 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.354895 2557 generic.go:356] "Generic (PLEG): container finished" podID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerID="31cd078820947023a7a4b2828944648777f125ab22a092c2b7487bcd4ca214c3" exitCode=0 Jul 20 15:33:01.355063 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.355023 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" event={"ID":"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e","Type":"ContainerDied","Data":"31cd078820947023a7a4b2828944648777f125ab22a092c2b7487bcd4ca214c3"} Jul 20 15:33:01.355127 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.355077 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" event={"ID":"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e","Type":"ContainerStarted","Data":"0176691fefa2ebf46a8e759cfbcd1a177306de96a70cd37d9e594d9953f5ee3e"} Jul 20 15:33:01.357355 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.357307 2557 generic.go:356] "Generic (PLEG): container finished" podID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerID="b3ecb949367ec1077f388659ab89a2efb8b434081041b7127fd2a02e88e8e6b6" exitCode=0 Jul 20 15:33:01.357484 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.357454 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" event={"ID":"be2760a3-1789-42bc-9f82-41044e67d6e7","Type":"ContainerDied","Data":"b3ecb949367ec1077f388659ab89a2efb8b434081041b7127fd2a02e88e8e6b6"} Jul 20 15:33:01.359969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.359947 2557 generic.go:356] "Generic (PLEG): container finished" podID="df101e46-4c7d-4942-864a-4587988ed46d" containerID="0483b26e83d6897183fd5dae72b7f3a5bc906c65d4aa26470fe412b8536750d3" exitCode=0 Jul 20 15:33:01.360075 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.360028 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" event={"ID":"df101e46-4c7d-4942-864a-4587988ed46d","Type":"ContainerDied","Data":"0483b26e83d6897183fd5dae72b7f3a5bc906c65d4aa26470fe412b8536750d3"} Jul 20 15:33:01.360075 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.360059 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" event={"ID":"df101e46-4c7d-4942-864a-4587988ed46d","Type":"ContainerStarted","Data":"4be64c80f160184a2561bacabb02c3d9d0c9f514c5b7914f4e999e22635df2d4"} Jul 20 15:33:01.402732 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:01.402710 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn"] Jul 20 15:33:01.404415 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:01.404387 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbadba61_e355_4b28_9725_a9b76ff809f7.slice/crio-be1de3f504001257e315487dd5ef0db3c510ae3fceb6877d5e6cb9f848657131 WatchSource:0}: Error finding container be1de3f504001257e315487dd5ef0db3c510ae3fceb6877d5e6cb9f848657131: Status 404 returned error can't find the container with id be1de3f504001257e315487dd5ef0db3c510ae3fceb6877d5e6cb9f848657131 Jul 20 15:33:02.368030 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.367954 2557 generic.go:356] "Generic (PLEG): container finished" podID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerID="7bc753633c9174f66bce11c41bde0dbcfe25a12dd51eebddea6d7dd8c84517ab" exitCode=0 Jul 20 15:33:02.368030 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.367996 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" event={"ID":"be2760a3-1789-42bc-9f82-41044e67d6e7","Type":"ContainerDied","Data":"7bc753633c9174f66bce11c41bde0dbcfe25a12dd51eebddea6d7dd8c84517ab"} Jul 20 15:33:02.369460 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.369439 2557 generic.go:356] "Generic (PLEG): container finished" podID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerID="e893e95d73a29d1c471ad4f83ae658d96924756e63fcc9e909395a647d1aefcb" exitCode=0 Jul 20 15:33:02.369585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.369522 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" event={"ID":"bbadba61-e355-4b28-9725-a9b76ff809f7","Type":"ContainerDied","Data":"e893e95d73a29d1c471ad4f83ae658d96924756e63fcc9e909395a647d1aefcb"} Jul 20 15:33:02.369585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.369572 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" event={"ID":"bbadba61-e355-4b28-9725-a9b76ff809f7","Type":"ContainerStarted","Data":"be1de3f504001257e315487dd5ef0db3c510ae3fceb6877d5e6cb9f848657131"} Jul 20 15:33:02.371712 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.371617 2557 generic.go:356] "Generic (PLEG): container finished" podID="df101e46-4c7d-4942-864a-4587988ed46d" containerID="f84408af678265567bb4473f9cb98339ed3c64f150299b62bee4a8f9fbd01903" exitCode=0 Jul 20 15:33:02.371712 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:02.371673 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" event={"ID":"df101e46-4c7d-4942-864a-4587988ed46d","Type":"ContainerDied","Data":"f84408af678265567bb4473f9cb98339ed3c64f150299b62bee4a8f9fbd01903"} Jul 20 15:33:03.379004 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.378966 2557 generic.go:356] "Generic (PLEG): container finished" podID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerID="9aee3605ad0e539d883ea127a10003be0182d3ab7e8bff264858418578eb7178" exitCode=0 Jul 20 15:33:03.379375 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.379053 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" event={"ID":"bbadba61-e355-4b28-9725-a9b76ff809f7","Type":"ContainerDied","Data":"9aee3605ad0e539d883ea127a10003be0182d3ab7e8bff264858418578eb7178"} Jul 20 15:33:03.381567 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.381537 2557 generic.go:356] "Generic (PLEG): container finished" podID="df101e46-4c7d-4942-864a-4587988ed46d" containerID="652b7c0b33c1f097aeabad809408cdfdf009ed6ed0681063cf1873b4f646076a" exitCode=0 Jul 20 15:33:03.381567 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.381547 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" event={"ID":"df101e46-4c7d-4942-864a-4587988ed46d","Type":"ContainerDied","Data":"652b7c0b33c1f097aeabad809408cdfdf009ed6ed0681063cf1873b4f646076a"} Jul 20 15:33:03.383604 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.383579 2557 generic.go:356] "Generic (PLEG): container finished" podID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerID="93b3cead71c8d912d6af57060621ff3d6c88eadcca0364bc67fe9f2729505de9" exitCode=0 Jul 20 15:33:03.383687 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.383667 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" event={"ID":"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e","Type":"ContainerDied","Data":"93b3cead71c8d912d6af57060621ff3d6c88eadcca0364bc67fe9f2729505de9"} Jul 20 15:33:03.539007 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.538981 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:33:03.634872 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.634846 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-util\") pod \"be2760a3-1789-42bc-9f82-41044e67d6e7\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " Jul 20 15:33:03.634988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.634916 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-bundle\") pod \"be2760a3-1789-42bc-9f82-41044e67d6e7\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " Jul 20 15:33:03.634988 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.634960 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9ffp\" (UniqueName: \"kubernetes.io/projected/be2760a3-1789-42bc-9f82-41044e67d6e7-kube-api-access-r9ffp\") pod \"be2760a3-1789-42bc-9f82-41044e67d6e7\" (UID: \"be2760a3-1789-42bc-9f82-41044e67d6e7\") " Jul 20 15:33:03.635479 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.635455 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-bundle" (OuterVolumeSpecName: "bundle") pod "be2760a3-1789-42bc-9f82-41044e67d6e7" (UID: "be2760a3-1789-42bc-9f82-41044e67d6e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:03.637015 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.636992 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2760a3-1789-42bc-9f82-41044e67d6e7-kube-api-access-r9ffp" (OuterVolumeSpecName: "kube-api-access-r9ffp") pod "be2760a3-1789-42bc-9f82-41044e67d6e7" (UID: "be2760a3-1789-42bc-9f82-41044e67d6e7"). InnerVolumeSpecName "kube-api-access-r9ffp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:33:03.640528 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.640499 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-util" (OuterVolumeSpecName: "util") pod "be2760a3-1789-42bc-9f82-41044e67d6e7" (UID: "be2760a3-1789-42bc-9f82-41044e67d6e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:03.736146 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.736114 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:03.736146 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.736142 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9ffp\" (UniqueName: \"kubernetes.io/projected/be2760a3-1789-42bc-9f82-41044e67d6e7-kube-api-access-r9ffp\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:03.736146 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:03.736152 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/be2760a3-1789-42bc-9f82-41044e67d6e7-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:04.390585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.390554 2557 generic.go:356] "Generic (PLEG): container finished" podID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerID="dcfdc215e15176ee60a38887b8e497abf0e6e857f537481be23c8438b70efff6" exitCode=0 Jul 20 15:33:04.391036 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.390633 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" event={"ID":"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e","Type":"ContainerDied","Data":"dcfdc215e15176ee60a38887b8e497abf0e6e857f537481be23c8438b70efff6"} Jul 20 15:33:04.392552 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.392532 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" Jul 20 15:33:04.392552 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.392545 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l" event={"ID":"be2760a3-1789-42bc-9f82-41044e67d6e7","Type":"ContainerDied","Data":"cef664a5a247bf7a1a11966bbf2fc187a8a7791fe4ecaf9fc6de78d5516ddc52"} Jul 20 15:33:04.392695 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.392580 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cef664a5a247bf7a1a11966bbf2fc187a8a7791fe4ecaf9fc6de78d5516ddc52" Jul 20 15:33:04.394688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.394666 2557 generic.go:356] "Generic (PLEG): container finished" podID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerID="6855b1913c215629ff064eb41b40a3e9b18f6e2a819169fe065a050e2a60492d" exitCode=0 Jul 20 15:33:04.394831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.394701 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" event={"ID":"bbadba61-e355-4b28-9725-a9b76ff809f7","Type":"ContainerDied","Data":"6855b1913c215629ff064eb41b40a3e9b18f6e2a819169fe065a050e2a60492d"} Jul 20 15:33:04.523476 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.523449 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:04.644544 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.644455 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr8q2\" (UniqueName: \"kubernetes.io/projected/df101e46-4c7d-4942-864a-4587988ed46d-kube-api-access-wr8q2\") pod \"df101e46-4c7d-4942-864a-4587988ed46d\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " Jul 20 15:33:04.644544 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.644539 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-util\") pod \"df101e46-4c7d-4942-864a-4587988ed46d\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " Jul 20 15:33:04.644801 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.644570 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-bundle\") pod \"df101e46-4c7d-4942-864a-4587988ed46d\" (UID: \"df101e46-4c7d-4942-864a-4587988ed46d\") " Jul 20 15:33:04.645260 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.645185 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-bundle" (OuterVolumeSpecName: "bundle") pod "df101e46-4c7d-4942-864a-4587988ed46d" (UID: "df101e46-4c7d-4942-864a-4587988ed46d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:04.646840 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.646809 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df101e46-4c7d-4942-864a-4587988ed46d-kube-api-access-wr8q2" (OuterVolumeSpecName: "kube-api-access-wr8q2") pod "df101e46-4c7d-4942-864a-4587988ed46d" (UID: "df101e46-4c7d-4942-864a-4587988ed46d"). InnerVolumeSpecName "kube-api-access-wr8q2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:33:04.652375 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.652330 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-util" (OuterVolumeSpecName: "util") pod "df101e46-4c7d-4942-864a-4587988ed46d" (UID: "df101e46-4c7d-4942-864a-4587988ed46d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:04.745571 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.745528 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:04.745571 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.745562 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df101e46-4c7d-4942-864a-4587988ed46d-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:04.745571 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:04.745576 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr8q2\" (UniqueName: \"kubernetes.io/projected/df101e46-4c7d-4942-864a-4587988ed46d-kube-api-access-wr8q2\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:05.401318 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.401286 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" Jul 20 15:33:05.401318 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.401294 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5" event={"ID":"df101e46-4c7d-4942-864a-4587988ed46d","Type":"ContainerDied","Data":"4be64c80f160184a2561bacabb02c3d9d0c9f514c5b7914f4e999e22635df2d4"} Jul 20 15:33:05.401841 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.401375 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4be64c80f160184a2561bacabb02c3d9d0c9f514c5b7914f4e999e22635df2d4" Jul 20 15:33:05.550999 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.550975 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:05.552801 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.552765 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:05.652788 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.652702 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4cst\" (UniqueName: \"kubernetes.io/projected/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-kube-api-access-s4cst\") pod \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " Jul 20 15:33:05.652788 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.652737 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-util\") pod \"bbadba61-e355-4b28-9725-a9b76ff809f7\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " Jul 20 15:33:05.652788 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.652760 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-bundle\") pod \"bbadba61-e355-4b28-9725-a9b76ff809f7\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " Jul 20 15:33:05.652995 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.652823 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvvm9\" (UniqueName: \"kubernetes.io/projected/bbadba61-e355-4b28-9725-a9b76ff809f7-kube-api-access-zvvm9\") pod \"bbadba61-e355-4b28-9725-a9b76ff809f7\" (UID: \"bbadba61-e355-4b28-9725-a9b76ff809f7\") " Jul 20 15:33:05.652995 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.652881 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-util\") pod \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " Jul 20 15:33:05.652995 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.652916 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-bundle\") pod \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\" (UID: \"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e\") " Jul 20 15:33:05.653340 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.653313 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-bundle" (OuterVolumeSpecName: "bundle") pod "bbadba61-e355-4b28-9725-a9b76ff809f7" (UID: "bbadba61-e355-4b28-9725-a9b76ff809f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:05.653822 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.653791 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-bundle" (OuterVolumeSpecName: "bundle") pod "99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" (UID: "99cfb23c-0d5d-4ffe-bb23-4d81dc85213e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:05.655095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.655071 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbadba61-e355-4b28-9725-a9b76ff809f7-kube-api-access-zvvm9" (OuterVolumeSpecName: "kube-api-access-zvvm9") pod "bbadba61-e355-4b28-9725-a9b76ff809f7" (UID: "bbadba61-e355-4b28-9725-a9b76ff809f7"). InnerVolumeSpecName "kube-api-access-zvvm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:33:05.655276 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.655256 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-kube-api-access-s4cst" (OuterVolumeSpecName: "kube-api-access-s4cst") pod "99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" (UID: "99cfb23c-0d5d-4ffe-bb23-4d81dc85213e"). InnerVolumeSpecName "kube-api-access-s4cst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:33:05.659241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.659220 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-util" (OuterVolumeSpecName: "util") pod "bbadba61-e355-4b28-9725-a9b76ff809f7" (UID: "bbadba61-e355-4b28-9725-a9b76ff809f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:05.660878 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.660850 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-util" (OuterVolumeSpecName: "util") pod "99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" (UID: "99cfb23c-0d5d-4ffe-bb23-4d81dc85213e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:05.753925 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.753895 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:05.753925 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.753922 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4cst\" (UniqueName: \"kubernetes.io/projected/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-kube-api-access-s4cst\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:05.754050 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.753936 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:05.754050 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.753948 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbadba61-e355-4b28-9725-a9b76ff809f7-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:05.754050 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.753959 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvvm9\" (UniqueName: \"kubernetes.io/projected/bbadba61-e355-4b28-9725-a9b76ff809f7-kube-api-access-zvvm9\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:05.754050 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:05.753970 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99cfb23c-0d5d-4ffe-bb23-4d81dc85213e-util\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:06.408072 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:06.408044 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" Jul 20 15:33:06.408525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:06.408070 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz" event={"ID":"99cfb23c-0d5d-4ffe-bb23-4d81dc85213e","Type":"ContainerDied","Data":"0176691fefa2ebf46a8e759cfbcd1a177306de96a70cd37d9e594d9953f5ee3e"} Jul 20 15:33:06.408525 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:06.408126 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0176691fefa2ebf46a8e759cfbcd1a177306de96a70cd37d9e594d9953f5ee3e" Jul 20 15:33:06.409968 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:06.409946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" event={"ID":"bbadba61-e355-4b28-9725-a9b76ff809f7","Type":"ContainerDied","Data":"be1de3f504001257e315487dd5ef0db3c510ae3fceb6877d5e6cb9f848657131"} Jul 20 15:33:06.409968 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:06.409962 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn" Jul 20 15:33:06.410111 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:06.409981 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be1de3f504001257e315487dd5ef0db3c510ae3fceb6877d5e6cb9f848657131" Jul 20 15:33:15.693619 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.693589 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm"] Jul 20 15:33:15.694004 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.693973 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="pull" Jul 20 15:33:15.694004 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.693986 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="pull" Jul 20 15:33:15.694004 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.693997 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="util" Jul 20 15:33:15.694004 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694003 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694011 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694017 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694026 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="pull" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694031 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="pull" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694040 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694045 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694054 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694059 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694068 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694073 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694078 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694084 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694091 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694096 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="util" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694101 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="pull" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694105 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="pull" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694117 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="pull" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694121 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="pull" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694130 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="extract" Jul 20 15:33:15.694137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694135 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="extract" Jul 20 15:33:15.694668 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694184 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df101e46-4c7d-4942-864a-4587988ed46d" containerName="extract" Jul 20 15:33:15.694668 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694192 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbadba61-e355-4b28-9725-a9b76ff809f7" containerName="extract" Jul 20 15:33:15.694668 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694201 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2760a3-1789-42bc-9f82-41044e67d6e7" containerName="extract" Jul 20 15:33:15.694668 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.694206 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cfb23c-0d5d-4ffe-bb23-4d81dc85213e" containerName="extract" Jul 20 15:33:15.700021 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.700005 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:15.702342 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.702321 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nzn87\"" Jul 20 15:33:15.702480 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.702425 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 15:33:15.715376 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.715353 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm"] Jul 20 15:33:15.844483 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.844445 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x6gt\" (UniqueName: \"kubernetes.io/projected/ea3a7ddc-69c0-4035-802f-eb48d1135353-kube-api-access-8x6gt\") pod \"dns-operator-controller-manager-864dd78c4c-srrvm\" (UID: \"ea3a7ddc-69c0-4035-802f-eb48d1135353\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:15.945565 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.945486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x6gt\" (UniqueName: \"kubernetes.io/projected/ea3a7ddc-69c0-4035-802f-eb48d1135353-kube-api-access-8x6gt\") pod \"dns-operator-controller-manager-864dd78c4c-srrvm\" (UID: \"ea3a7ddc-69c0-4035-802f-eb48d1135353\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:15.957196 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:15.957170 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x6gt\" (UniqueName: \"kubernetes.io/projected/ea3a7ddc-69c0-4035-802f-eb48d1135353-kube-api-access-8x6gt\") pod \"dns-operator-controller-manager-864dd78c4c-srrvm\" (UID: \"ea3a7ddc-69c0-4035-802f-eb48d1135353\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:16.010399 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:16.010371 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:16.147033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:16.147001 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm"] Jul 20 15:33:16.148817 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:16.148794 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3a7ddc_69c0_4035_802f_eb48d1135353.slice/crio-23733d1c8fdf89ef56da1a2211c05178c4aad2e4ffb196be58586560d84e9bd8 WatchSource:0}: Error finding container 23733d1c8fdf89ef56da1a2211c05178c4aad2e4ffb196be58586560d84e9bd8: Status 404 returned error can't find the container with id 23733d1c8fdf89ef56da1a2211c05178c4aad2e4ffb196be58586560d84e9bd8 Jul 20 15:33:16.465153 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:16.465121 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" event={"ID":"ea3a7ddc-69c0-4035-802f-eb48d1135353","Type":"ContainerStarted","Data":"23733d1c8fdf89ef56da1a2211c05178c4aad2e4ffb196be58586560d84e9bd8"} Jul 20 15:33:18.478442 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:18.478404 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" event={"ID":"ea3a7ddc-69c0-4035-802f-eb48d1135353","Type":"ContainerStarted","Data":"7fc973367c21c49ab1ca0123b687ca7c4e12a6fb0a94545d94e2b466dc190897"} Jul 20 15:33:18.478847 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:18.478464 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:18.497578 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:18.497504 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" podStartSLOduration=1.242786072 podStartE2EDuration="3.497486855s" totalImagesPullingTime="2.254700783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:18.49658614 +0000 UTC m=+390.441179510" watchObservedRunningTime="2026-07-20 15:33:18.497486855 +0000 UTC m=+390.442080224" Jul 20 15:33:19.164484 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.164445 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr"] Jul 20 15:33:19.168075 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.168057 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:19.170345 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.170323 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xjb62\"" Jul 20 15:33:19.179231 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.179210 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr"] Jul 20 15:33:19.274214 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.274179 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhbm\" (UniqueName: \"kubernetes.io/projected/34a000ad-9a74-48e3-aab3-ea1ee149b140-kube-api-access-nzhbm\") pod \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" (UID: \"34a000ad-9a74-48e3-aab3-ea1ee149b140\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:19.374963 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.374931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhbm\" (UniqueName: \"kubernetes.io/projected/34a000ad-9a74-48e3-aab3-ea1ee149b140-kube-api-access-nzhbm\") pod \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" (UID: \"34a000ad-9a74-48e3-aab3-ea1ee149b140\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:19.396219 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.396195 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhbm\" (UniqueName: \"kubernetes.io/projected/34a000ad-9a74-48e3-aab3-ea1ee149b140-kube-api-access-nzhbm\") pod \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" (UID: \"34a000ad-9a74-48e3-aab3-ea1ee149b140\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:19.478872 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.478792 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:19.615341 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:19.615316 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr"] Jul 20 15:33:19.617077 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:19.617049 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a000ad_9a74_48e3_aab3_ea1ee149b140.slice/crio-ecc11d77e7252d81aa815c7a62b349f2a69c7bc6d3980d2e4478deaa8f0ff031 WatchSource:0}: Error finding container ecc11d77e7252d81aa815c7a62b349f2a69c7bc6d3980d2e4478deaa8f0ff031: Status 404 returned error can't find the container with id ecc11d77e7252d81aa815c7a62b349f2a69c7bc6d3980d2e4478deaa8f0ff031 Jul 20 15:33:20.489917 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:20.489887 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" event={"ID":"34a000ad-9a74-48e3-aab3-ea1ee149b140","Type":"ContainerStarted","Data":"ecc11d77e7252d81aa815c7a62b349f2a69c7bc6d3980d2e4478deaa8f0ff031"} Jul 20 15:33:21.497326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:21.497287 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" event={"ID":"34a000ad-9a74-48e3-aab3-ea1ee149b140","Type":"ContainerStarted","Data":"dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07"} Jul 20 15:33:21.497809 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:21.497406 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:21.524354 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:21.524274 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" podStartSLOduration=0.877483417 podStartE2EDuration="2.524259215s" totalImagesPullingTime="1.646775798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:21.522235521 +0000 UTC m=+393.466828889" watchObservedRunningTime="2026-07-20 15:33:21.524259215 +0000 UTC m=+393.468852584" Jul 20 15:33:24.154478 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.154435 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx"] Jul 20 15:33:24.157035 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.157013 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.159126 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.159108 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zdfnq\"" Jul 20 15:33:24.170697 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.170674 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx"] Jul 20 15:33:24.210440 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.210395 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deff0006-cf6b-46e9-ac3b-3038639ec4c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.210619 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.210540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q27cp\" (UniqueName: \"kubernetes.io/projected/deff0006-cf6b-46e9-ac3b-3038639ec4c0-kube-api-access-q27cp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.311534 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.311500 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deff0006-cf6b-46e9-ac3b-3038639ec4c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.311714 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.311599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q27cp\" (UniqueName: \"kubernetes.io/projected/deff0006-cf6b-46e9-ac3b-3038639ec4c0-kube-api-access-q27cp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.311942 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.311923 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deff0006-cf6b-46e9-ac3b-3038639ec4c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.328846 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.328824 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q27cp\" (UniqueName: \"kubernetes.io/projected/deff0006-cf6b-46e9-ac3b-3038639ec4c0-kube-api-access-q27cp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.466684 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.466598 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:24.612094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:24.612070 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx"] Jul 20 15:33:24.614272 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:24.614244 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeff0006_cf6b_46e9_ac3b_3038639ec4c0.slice/crio-0161cd953e6368afa5e3e75c13569755987bbe0fc3a046625c159065906d40e3 WatchSource:0}: Error finding container 0161cd953e6368afa5e3e75c13569755987bbe0fc3a046625c159065906d40e3: Status 404 returned error can't find the container with id 0161cd953e6368afa5e3e75c13569755987bbe0fc3a046625c159065906d40e3 Jul 20 15:33:25.522989 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:25.522945 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" event={"ID":"deff0006-cf6b-46e9-ac3b-3038639ec4c0","Type":"ContainerStarted","Data":"0161cd953e6368afa5e3e75c13569755987bbe0fc3a046625c159065906d40e3"} Jul 20 15:33:27.381466 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.381426 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q4htx"] Jul 20 15:33:27.384734 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.384714 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:27.386807 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.386765 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-mgv2q\"" Jul 20 15:33:27.395291 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.395268 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q4htx"] Jul 20 15:33:27.443848 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.443816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7znln\" (UniqueName: \"kubernetes.io/projected/413ad218-87ae-485e-bf0a-4a3ad7f2b07c-kube-api-access-7znln\") pod \"authorino-operator-75cc8d5cb7-q4htx\" (UID: \"413ad218-87ae-485e-bf0a-4a3ad7f2b07c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:27.544533 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.544512 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7znln\" (UniqueName: \"kubernetes.io/projected/413ad218-87ae-485e-bf0a-4a3ad7f2b07c-kube-api-access-7znln\") pod \"authorino-operator-75cc8d5cb7-q4htx\" (UID: \"413ad218-87ae-485e-bf0a-4a3ad7f2b07c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:27.553606 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.553573 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7znln\" (UniqueName: \"kubernetes.io/projected/413ad218-87ae-485e-bf0a-4a3ad7f2b07c-kube-api-access-7znln\") pod \"authorino-operator-75cc8d5cb7-q4htx\" (UID: \"413ad218-87ae-485e-bf0a-4a3ad7f2b07c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:27.697474 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.697371 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:27.828893 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:27.828764 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q4htx"] Jul 20 15:33:27.831265 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:27.831233 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413ad218_87ae_485e_bf0a_4a3ad7f2b07c.slice/crio-bb4221463b300691665213d6333fec91c57ec614660e8eb0520f77f8839de67a WatchSource:0}: Error finding container bb4221463b300691665213d6333fec91c57ec614660e8eb0520f77f8839de67a: Status 404 returned error can't find the container with id bb4221463b300691665213d6333fec91c57ec614660e8eb0520f77f8839de67a Jul 20 15:33:28.545214 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:28.545180 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" event={"ID":"413ad218-87ae-485e-bf0a-4a3ad7f2b07c","Type":"ContainerStarted","Data":"bb4221463b300691665213d6333fec91c57ec614660e8eb0520f77f8839de67a"} Jul 20 15:33:29.486354 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:29.486323 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-srrvm" Jul 20 15:33:31.566342 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:31.566262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" event={"ID":"deff0006-cf6b-46e9-ac3b-3038639ec4c0","Type":"ContainerStarted","Data":"817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28"} Jul 20 15:33:31.566753 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:31.566474 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:31.567761 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:31.567737 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" event={"ID":"413ad218-87ae-485e-bf0a-4a3ad7f2b07c","Type":"ContainerStarted","Data":"d74f9de64883d6372f0729be6037e7b656c6ba966475df55501398344709f474"} Jul 20 15:33:31.567892 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:31.567875 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:31.584942 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:31.584892 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" podStartSLOduration=0.93923703 podStartE2EDuration="7.584879464s" totalImagesPullingTime="6.645642434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:31.583702356 +0000 UTC m=+403.528295725" watchObservedRunningTime="2026-07-20 15:33:31.584879464 +0000 UTC m=+403.529472893" Jul 20 15:33:31.602390 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:31.602344 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" podStartSLOduration=1.186509284 podStartE2EDuration="4.602332237s" totalImagesPullingTime="3.415822953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:31.600349046 +0000 UTC m=+403.544942416" watchObservedRunningTime="2026-07-20 15:33:31.602332237 +0000 UTC m=+403.546925607" Jul 20 15:33:32.505022 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:32.504984 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:33.897681 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:33.897647 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb"] Jul 20 15:33:33.901200 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:33.901182 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:33.903634 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:33.903598 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x6bxm\"" Jul 20 15:33:33.903634 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:33.903626 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 15:33:33.903850 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:33.903650 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 15:33:33.909683 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:33.909661 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb"] Jul 20 15:33:34.007575 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.007549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x64t\" (UniqueName: \"kubernetes.io/projected/e571d3cd-7d57-4742-b0b6-1949be6e4496-kube-api-access-9x64t\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.007721 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.007596 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e571d3cd-7d57-4742-b0b6-1949be6e4496-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.007721 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.007616 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e571d3cd-7d57-4742-b0b6-1949be6e4496-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.108962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.108936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x64t\" (UniqueName: \"kubernetes.io/projected/e571d3cd-7d57-4742-b0b6-1949be6e4496-kube-api-access-9x64t\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.109093 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.108983 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e571d3cd-7d57-4742-b0b6-1949be6e4496-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.109093 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.109004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e571d3cd-7d57-4742-b0b6-1949be6e4496-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.109093 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:34.109076 2557 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 20 15:33:34.109195 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:34.109130 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e571d3cd-7d57-4742-b0b6-1949be6e4496-plugin-serving-cert podName:e571d3cd-7d57-4742-b0b6-1949be6e4496 nodeName:}" failed. No retries permitted until 2026-07-20 15:33:34.609117115 +0000 UTC m=+406.553710463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e571d3cd-7d57-4742-b0b6-1949be6e4496-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-xg8mb" (UID: "e571d3cd-7d57-4742-b0b6-1949be6e4496") : secret "plugin-serving-cert" not found Jul 20 15:33:34.109637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.109621 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e571d3cd-7d57-4742-b0b6-1949be6e4496-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.117073 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.117054 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x64t\" (UniqueName: \"kubernetes.io/projected/e571d3cd-7d57-4742-b0b6-1949be6e4496-kube-api-access-9x64t\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.612368 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.612338 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e571d3cd-7d57-4742-b0b6-1949be6e4496-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.614931 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.614905 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e571d3cd-7d57-4742-b0b6-1949be6e4496-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xg8mb\" (UID: \"e571d3cd-7d57-4742-b0b6-1949be6e4496\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.811087 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.811049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" Jul 20 15:33:34.940665 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:34.940637 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb"] Jul 20 15:33:34.942381 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:34.942357 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode571d3cd_7d57_4742_b0b6_1949be6e4496.slice/crio-440e03dad5105361be87b85155a9dca51efdea673df181e94827d6695f152c91 WatchSource:0}: Error finding container 440e03dad5105361be87b85155a9dca51efdea673df181e94827d6695f152c91: Status 404 returned error can't find the container with id 440e03dad5105361be87b85155a9dca51efdea673df181e94827d6695f152c91 Jul 20 15:33:35.589208 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:35.589172 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" event={"ID":"e571d3cd-7d57-4742-b0b6-1949be6e4496","Type":"ContainerStarted","Data":"440e03dad5105361be87b85155a9dca51efdea673df181e94827d6695f152c91"} Jul 20 15:33:42.574605 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:42.574569 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q4htx" Jul 20 15:33:42.574992 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:42.574730 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:44.061823 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.061780 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt"] Jul 20 15:33:44.069736 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.069707 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.079641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.079617 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt"] Jul 20 15:33:44.201741 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.201701 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7cd9\" (UniqueName: \"kubernetes.io/projected/a8c2b345-b458-4430-b364-3fa07c78052c-kube-api-access-c7cd9\") pod \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.201932 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.201859 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c2b345-b458-4430-b364-3fa07c78052c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.303068 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.303028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c2b345-b458-4430-b364-3fa07c78052c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.303241 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.303097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7cd9\" (UniqueName: \"kubernetes.io/projected/a8c2b345-b458-4430-b364-3fa07c78052c-kube-api-access-c7cd9\") pod \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.303432 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.303411 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c2b345-b458-4430-b364-3fa07c78052c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.311990 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.311925 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7cd9\" (UniqueName: \"kubernetes.io/projected/a8c2b345-b458-4430-b364-3fa07c78052c-kube-api-access-c7cd9\") pod \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.383834 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.383800 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:44.490935 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.490898 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt"] Jul 20 15:33:44.494379 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.494354 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt"] Jul 20 15:33:44.518464 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.518325 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx"] Jul 20 15:33:44.518831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.518556 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" containerName="manager" containerID="cri-o://817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28" gracePeriod=2 Jul 20 15:33:44.520581 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:44.520546 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:44.520724 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.520637 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx"] Jul 20 15:33:44.540942 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.540917 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m"] Jul 20 15:33:44.541450 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.541435 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" containerName="manager" Jul 20 15:33:44.541538 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.541454 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" containerName="manager" Jul 20 15:33:44.541602 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.541558 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" containerName="manager" Jul 20 15:33:44.543842 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.543824 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.546205 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:44.546179 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:44.555486 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.555152 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr"] Jul 20 15:33:44.555486 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.555333 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" containerName="manager" containerID="cri-o://dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07" gracePeriod=2 Jul 20 15:33:44.569707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.569405 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc"] Jul 20 15:33:44.572380 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.572363 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.574462 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.574440 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m"] Jul 20 15:33:44.578088 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.578064 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr"] Jul 20 15:33:44.585562 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.585528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc"] Jul 20 15:33:44.600926 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:44.600899 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:44.605399 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.605371 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckxh8\" (UniqueName: \"kubernetes.io/projected/3e36e1b3-af13-4e0a-958e-b90b85164881-kube-api-access-ckxh8\") pod \"kuadrant-operator-controller-manager-55467d9f7-d484m\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.605476 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.605422 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e36e1b3-af13-4e0a-958e-b90b85164881-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-d484m\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.636041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.634473 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx"] Jul 20 15:33:44.636041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.635045 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" containerName="manager" Jul 20 15:33:44.636041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.635065 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" containerName="manager" Jul 20 15:33:44.636041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.635199 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" containerName="manager" Jul 20 15:33:44.638037 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.637922 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:33:44.651036 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.650447 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx"] Jul 20 15:33:44.666498 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:44.666464 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:44.706148 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.706113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckxh8\" (UniqueName: \"kubernetes.io/projected/3e36e1b3-af13-4e0a-958e-b90b85164881-kube-api-access-ckxh8\") pod \"kuadrant-operator-controller-manager-55467d9f7-d484m\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.706255 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.706191 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e36e1b3-af13-4e0a-958e-b90b85164881-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-d484m\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.706324 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.706256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmwdg\" (UniqueName: \"kubernetes.io/projected/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-kube-api-access-nmwdg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xl9wc\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.706324 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.706293 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl85m\" (UniqueName: \"kubernetes.io/projected/5ba8fb3b-925d-498d-9bbb-0e18a7541bc3-kube-api-access-xl85m\") pod \"limitador-operator-controller-manager-859d7b78bd-znpxx\" (UID: \"5ba8fb3b-925d-498d-9bbb-0e18a7541bc3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:33:44.706429 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.706321 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xl9wc\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.706625 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.706599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e36e1b3-af13-4e0a-958e-b90b85164881-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-d484m\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.714748 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.714726 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckxh8\" (UniqueName: \"kubernetes.io/projected/3e36e1b3-af13-4e0a-958e-b90b85164881-kube-api-access-ckxh8\") pod \"kuadrant-operator-controller-manager-55467d9f7-d484m\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.807445 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.807416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmwdg\" (UniqueName: \"kubernetes.io/projected/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-kube-api-access-nmwdg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xl9wc\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.807583 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.807466 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl85m\" (UniqueName: \"kubernetes.io/projected/5ba8fb3b-925d-498d-9bbb-0e18a7541bc3-kube-api-access-xl85m\") pod \"limitador-operator-controller-manager-859d7b78bd-znpxx\" (UID: \"5ba8fb3b-925d-498d-9bbb-0e18a7541bc3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:33:44.807583 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.807520 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xl9wc\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.807983 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.807961 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xl9wc\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.818454 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.818397 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmwdg\" (UniqueName: \"kubernetes.io/projected/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-kube-api-access-nmwdg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xl9wc\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.818983 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.818959 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl85m\" (UniqueName: \"kubernetes.io/projected/5ba8fb3b-925d-498d-9bbb-0e18a7541bc3-kube-api-access-xl85m\") pod \"limitador-operator-controller-manager-859d7b78bd-znpxx\" (UID: \"5ba8fb3b-925d-498d-9bbb-0e18a7541bc3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:33:44.935941 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.935906 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:44.945471 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.945447 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:44.955350 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:44.955328 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:33:56.969286 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:56.968966 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:56.971041 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:56.971004 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:56.990897 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:56.990637 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:56.992729 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:56.992611 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:56.994315 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:56.994269 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:57.035388 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.035353 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m"] Jul 20 15:33:57.040420 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:57.040385 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e36e1b3_af13_4e0a_958e_b90b85164881.slice/crio-62a58cba358ebee4bcb68cb071bce2b34fe15aedeab0a4bb32d95fed3b11b68c WatchSource:0}: Error finding container 62a58cba358ebee4bcb68cb071bce2b34fe15aedeab0a4bb32d95fed3b11b68c: Status 404 returned error can't find the container with id 62a58cba358ebee4bcb68cb071bce2b34fe15aedeab0a4bb32d95fed3b11b68c Jul 20 15:33:57.127432 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.127398 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q27cp\" (UniqueName: \"kubernetes.io/projected/deff0006-cf6b-46e9-ac3b-3038639ec4c0-kube-api-access-q27cp\") pod \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " Jul 20 15:33:57.127432 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.127437 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzhbm\" (UniqueName: \"kubernetes.io/projected/34a000ad-9a74-48e3-aab3-ea1ee149b140-kube-api-access-nzhbm\") pod \"34a000ad-9a74-48e3-aab3-ea1ee149b140\" (UID: \"34a000ad-9a74-48e3-aab3-ea1ee149b140\") " Jul 20 15:33:57.127691 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.127499 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deff0006-cf6b-46e9-ac3b-3038639ec4c0-extensions-socket-volume\") pod \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\" (UID: \"deff0006-cf6b-46e9-ac3b-3038639ec4c0\") " Jul 20 15:33:57.128329 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.128301 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deff0006-cf6b-46e9-ac3b-3038639ec4c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "deff0006-cf6b-46e9-ac3b-3038639ec4c0" (UID: "deff0006-cf6b-46e9-ac3b-3038639ec4c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:57.129609 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.129582 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deff0006-cf6b-46e9-ac3b-3038639ec4c0-kube-api-access-q27cp" (OuterVolumeSpecName: "kube-api-access-q27cp") pod "deff0006-cf6b-46e9-ac3b-3038639ec4c0" (UID: "deff0006-cf6b-46e9-ac3b-3038639ec4c0"). InnerVolumeSpecName "kube-api-access-q27cp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:33:57.129706 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.129677 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a000ad-9a74-48e3-aab3-ea1ee149b140-kube-api-access-nzhbm" (OuterVolumeSpecName: "kube-api-access-nzhbm") pod "34a000ad-9a74-48e3-aab3-ea1ee149b140" (UID: "34a000ad-9a74-48e3-aab3-ea1ee149b140"). InnerVolumeSpecName "kube-api-access-nzhbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:33:57.228254 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.228222 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q27cp\" (UniqueName: \"kubernetes.io/projected/deff0006-cf6b-46e9-ac3b-3038639ec4c0-kube-api-access-q27cp\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:57.228254 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.228251 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzhbm\" (UniqueName: \"kubernetes.io/projected/34a000ad-9a74-48e3-aab3-ea1ee149b140-kube-api-access-nzhbm\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:57.228254 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.228260 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deff0006-cf6b-46e9-ac3b-3038639ec4c0-extensions-socket-volume\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:33:57.258020 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.257990 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx"] Jul 20 15:33:57.260554 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:57.260530 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba8fb3b_925d_498d_9bbb_0e18a7541bc3.slice/crio-5e601e77452177b449dd15e8550594aa812512977781be6698072e753a208cbb WatchSource:0}: Error finding container 5e601e77452177b449dd15e8550594aa812512977781be6698072e753a208cbb: Status 404 returned error can't find the container with id 5e601e77452177b449dd15e8550594aa812512977781be6698072e753a208cbb Jul 20 15:33:57.261326 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.261301 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc"] Jul 20 15:33:57.263129 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:57.263109 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9136137_9f7c_4c06_a5c8_00d8d071b9d9.slice/crio-5b0afda280f79e6e1029c9584d2079753e4f5516dec78497debecf1b32c3590f WatchSource:0}: Error finding container 5b0afda280f79e6e1029c9584d2079753e4f5516dec78497debecf1b32c3590f: Status 404 returned error can't find the container with id 5b0afda280f79e6e1029c9584d2079753e4f5516dec78497debecf1b32c3590f Jul 20 15:33:57.736616 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.736584 2557 generic.go:356] "Generic (PLEG): container finished" podID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" containerID="817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28" exitCode=0 Jul 20 15:33:57.736847 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.736656 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:57.737548 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.736683 2557 scope.go:117] "RemoveContainer" containerID="817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28" Jul 20 15:33:57.738151 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.738127 2557 generic.go:356] "Generic (PLEG): container finished" podID="34a000ad-9a74-48e3-aab3-ea1ee149b140" containerID="dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07" exitCode=0 Jul 20 15:33:57.738245 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.738178 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:57.738682 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.738660 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:57.740110 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.740062 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:57.740214 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.740141 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" event={"ID":"3e36e1b3-af13-4e0a-958e-b90b85164881","Type":"ContainerStarted","Data":"9bf59ecb69e8374e4a90fcb5b44fdeb5bffe8e5e68efdb04b8a53fb796c6dfc5"} Jul 20 15:33:57.740269 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.740253 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:33:57.740302 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.740280 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" event={"ID":"3e36e1b3-af13-4e0a-958e-b90b85164881","Type":"ContainerStarted","Data":"62a58cba358ebee4bcb68cb071bce2b34fe15aedeab0a4bb32d95fed3b11b68c"} Jul 20 15:33:57.741498 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.741475 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:57.741676 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.741653 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" event={"ID":"e571d3cd-7d57-4742-b0b6-1949be6e4496","Type":"ContainerStarted","Data":"a55a175cb62bf976c39abe0c674007b687d03743671a0b38f2cb0311ef5b40d3"} Jul 20 15:33:57.742828 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.742802 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:57.743667 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.743639 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" event={"ID":"5ba8fb3b-925d-498d-9bbb-0e18a7541bc3","Type":"ContainerStarted","Data":"747a67f213da01897fde6a7fdf6d1234032097515fb06f04ca092acb033fb15b"} Jul 20 15:33:57.743757 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.743688 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:33:57.743757 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.743712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" event={"ID":"5ba8fb3b-925d-498d-9bbb-0e18a7541bc3","Type":"ContainerStarted","Data":"5e601e77452177b449dd15e8550594aa812512977781be6698072e753a208cbb"} Jul 20 15:33:57.744032 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.744008 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:57.745234 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.745208 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:57.745467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.745443 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" event={"ID":"a9136137-9f7c-4c06-a5c8-00d8d071b9d9","Type":"ContainerStarted","Data":"476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af"} Jul 20 15:33:57.745532 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.745491 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" event={"ID":"a9136137-9f7c-4c06-a5c8-00d8d071b9d9","Type":"ContainerStarted","Data":"5b0afda280f79e6e1029c9584d2079753e4f5516dec78497debecf1b32c3590f"} Jul 20 15:33:57.745591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.745573 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:33:57.748751 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.748731 2557 scope.go:117] "RemoveContainer" containerID="817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28" Jul 20 15:33:57.749055 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.749028 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28\": container with ID starting with 817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28 not found: ID does not exist" containerID="817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28" Jul 20 15:33:57.749130 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.749059 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28"} err="failed to get container status \"817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28\": rpc error: code = NotFound desc = could not find container \"817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28\": container with ID starting with 817c2f4e4e9c6766415a4a81d2c93574ba5bcd6633209df9e604e1939f8cbc28 not found: ID does not exist" Jul 20 15:33:57.749130 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.749079 2557 scope.go:117] "RemoveContainer" containerID="dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07" Jul 20 15:33:57.757135 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.757117 2557 scope.go:117] "RemoveContainer" containerID="dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07" Jul 20 15:33:57.757368 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.757350 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07\": container with ID starting with dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07 not found: ID does not exist" containerID="dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07" Jul 20 15:33:57.757408 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.757376 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07"} err="failed to get container status \"dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07\": rpc error: code = NotFound desc = could not find container \"dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07\": container with ID starting with dd95975b31dd3be4403cdadb5fe00e775880346ca629817d41c1bb4233350d07 not found: ID does not exist" Jul 20 15:33:57.764992 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.764952 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" podStartSLOduration=13.764936962 podStartE2EDuration="13.764936962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:57.762296942 +0000 UTC m=+429.706890313" watchObservedRunningTime="2026-07-20 15:33:57.764936962 +0000 UTC m=+429.709530333" Jul 20 15:33:57.780091 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.780040 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xg8mb" podStartSLOduration=2.824139801 podStartE2EDuration="24.780026165s" totalImagesPullingTime="21.955886364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:57.776488872 +0000 UTC m=+429.721082241" watchObservedRunningTime="2026-07-20 15:33:57.780026165 +0000 UTC m=+429.724619539" Jul 20 15:33:57.798477 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.798442 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:57.799356 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.799306 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" podStartSLOduration=13.799286174 podStartE2EDuration="13.799286174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:57.796818755 +0000 UTC m=+429.741412126" watchObservedRunningTime="2026-07-20 15:33:57.799286174 +0000 UTC m=+429.743879546" Jul 20 15:33:57.800330 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:57.800305 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:57.821438 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:57.821287 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" podStartSLOduration=13.821265403 podStartE2EDuration="13.821265403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:57.819282619 +0000 UTC m=+429.763875992" watchObservedRunningTime="2026-07-20 15:33:57.821265403 +0000 UTC m=+429.765858774" Jul 20 15:33:58.094497 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.094403 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69dfb79d48-kpgvf"] Jul 20 15:33:58.097588 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.097570 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.114551 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.114515 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69dfb79d48-kpgvf"] Jul 20 15:33:58.141591 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:58.141547 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:58.143189 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:58.143165 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:58.237183 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237144 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-oauth-serving-cert\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.237183 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237187 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h8tx\" (UniqueName: \"kubernetes.io/projected/6b11b8be-4eba-4435-b215-5e84eb2d0e73-kube-api-access-9h8tx\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.237413 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237238 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-service-ca\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.237413 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237315 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-serving-cert\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.237413 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237362 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-oauth-config\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.237413 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237390 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-config\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.237556 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.237423 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-trusted-ca-bundle\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338510 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-service-ca\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338536 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-serving-cert\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-oauth-config\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-config\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-trusted-ca-bundle\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338688 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-oauth-serving-cert\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.338994 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.338710 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h8tx\" (UniqueName: \"kubernetes.io/projected/6b11b8be-4eba-4435-b215-5e84eb2d0e73-kube-api-access-9h8tx\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.339380 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.339350 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-service-ca\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.339494 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.339365 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-config\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.339494 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.339428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-oauth-serving-cert\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.339567 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.339518 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b11b8be-4eba-4435-b215-5e84eb2d0e73-trusted-ca-bundle\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.341730 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.341708 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-oauth-config\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.341811 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.341801 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b11b8be-4eba-4435-b215-5e84eb2d0e73-console-serving-cert\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.346957 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.346906 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h8tx\" (UniqueName: \"kubernetes.io/projected/6b11b8be-4eba-4435-b215-5e84eb2d0e73-kube-api-access-9h8tx\") pod \"console-69dfb79d48-kpgvf\" (UID: \"6b11b8be-4eba-4435-b215-5e84eb2d0e73\") " pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.406391 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.406340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:33:58.550949 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.550926 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69dfb79d48-kpgvf"] Jul 20 15:33:58.552999 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:33:58.552973 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b11b8be_4eba_4435_b215_5e84eb2d0e73.slice/crio-044215dc93777801b2d27e91ab2ea594a9b4c9024c9c5001da4e9f917030fbd7 WatchSource:0}: Error finding container 044215dc93777801b2d27e91ab2ea594a9b4c9024c9c5001da4e9f917030fbd7: Status 404 returned error can't find the container with id 044215dc93777801b2d27e91ab2ea594a9b4c9024c9c5001da4e9f917030fbd7 Jul 20 15:33:58.563642 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:58.563608 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-97jfx\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-97jfx" Jul 20 15:33:58.566961 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:58.566932 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8nbr\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8nbr" Jul 20 15:33:58.567242 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.567219 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34a000ad-9a74-48e3-aab3-ea1ee149b140" path="/var/lib/kubelet/pods/34a000ad-9a74-48e3-aab3-ea1ee149b140/volumes" Jul 20 15:33:58.567746 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.567730 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deff0006-cf6b-46e9-ac3b-3038639ec4c0" path="/var/lib/kubelet/pods/deff0006-cf6b-46e9-ac3b-3038639ec4c0/volumes" Jul 20 15:33:58.753182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.753146 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69dfb79d48-kpgvf" event={"ID":"6b11b8be-4eba-4435-b215-5e84eb2d0e73","Type":"ContainerStarted","Data":"03738fbeae11762cc7436b70403e40ebe721324d949e753bf2acc722ecb8956c"} Jul 20 15:33:58.753387 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.753199 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69dfb79d48-kpgvf" event={"ID":"6b11b8be-4eba-4435-b215-5e84eb2d0e73","Type":"ContainerStarted","Data":"044215dc93777801b2d27e91ab2ea594a9b4c9024c9c5001da4e9f917030fbd7"} Jul 20 15:33:58.776040 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:58.775980 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69dfb79d48-kpgvf" podStartSLOduration=0.775957642 podStartE2EDuration="775.957642ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:33:58.774541251 +0000 UTC m=+430.719134623" watchObservedRunningTime="2026-07-20 15:33:58.775957642 +0000 UTC m=+430.720551014" Jul 20 15:33:59.388280 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:59.388243 2557 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5hlkt_kuadrant-system_a8c2b345-b458-4430-b364-3fa07c78052c_0(fb5ea559817428343dc99688d5adeab089a9b5e6c47082ecbaf09b2d96212b0d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5hlkt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 20 15:33:59.388545 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:59.388309 2557 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5hlkt_kuadrant-system_a8c2b345-b458-4430-b364-3fa07c78052c_0(fb5ea559817428343dc99688d5adeab089a9b5e6c47082ecbaf09b2d96212b0d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5hlkt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:59.760718 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:59.760627 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:59.762563 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:59.762537 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="a8c2b345-b458-4430-b364-3fa07c78052c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:59.767329 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:59.767307 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:59.769696 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:33:59.769670 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="a8c2b345-b458-4430-b364-3fa07c78052c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:33:59.955141 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:59.955104 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c2b345-b458-4430-b364-3fa07c78052c-extensions-socket-volume\") pod \"a8c2b345-b458-4430-b364-3fa07c78052c\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " Jul 20 15:33:59.955309 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:59.955154 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7cd9\" (UniqueName: \"kubernetes.io/projected/a8c2b345-b458-4430-b364-3fa07c78052c-kube-api-access-c7cd9\") pod \"a8c2b345-b458-4430-b364-3fa07c78052c\" (UID: \"a8c2b345-b458-4430-b364-3fa07c78052c\") " Jul 20 15:33:59.955406 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:59.955381 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c2b345-b458-4430-b364-3fa07c78052c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a8c2b345-b458-4430-b364-3fa07c78052c" (UID: "a8c2b345-b458-4430-b364-3fa07c78052c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:33:59.957502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:33:59.957478 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c2b345-b458-4430-b364-3fa07c78052c-kube-api-access-c7cd9" (OuterVolumeSpecName: "kube-api-access-c7cd9") pod "a8c2b345-b458-4430-b364-3fa07c78052c" (UID: "a8c2b345-b458-4430-b364-3fa07c78052c"). InnerVolumeSpecName "kube-api-access-c7cd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:34:00.058607 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:00.056091 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8c2b345-b458-4430-b364-3fa07c78052c-extensions-socket-volume\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:00.058607 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:00.056126 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7cd9\" (UniqueName: \"kubernetes.io/projected/a8c2b345-b458-4430-b364-3fa07c78052c-kube-api-access-c7cd9\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:00.566203 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:00.566163 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c2b345-b458-4430-b364-3fa07c78052c" path="/var/lib/kubelet/pods/a8c2b345-b458-4430-b364-3fa07c78052c/volumes" Jul 20 15:34:00.766353 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:00.766321 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:34:00.768338 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:34:00.768307 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="a8c2b345-b458-4430-b364-3fa07c78052c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:34:00.769913 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:34:00.769890 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5hlkt\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="a8c2b345-b458-4430-b364-3fa07c78052c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5hlkt" Jul 20 15:34:08.407311 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.407273 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:34:08.407743 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.407346 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:34:08.412023 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.412000 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:34:08.756369 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.756277 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:34:08.756608 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.756587 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:34:08.756678 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.756657 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-znpxx" Jul 20 15:34:08.820910 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.820880 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69dfb79d48-kpgvf" Jul 20 15:34:08.841406 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.841366 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc"] Jul 20 15:34:08.841595 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.841536 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" podUID="a9136137-9f7c-4c06-a5c8-00d8d071b9d9" containerName="manager" containerID="cri-o://476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af" gracePeriod=10 Jul 20 15:34:08.887212 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:08.887177 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b8959f7c6-47kcb"] Jul 20 15:34:09.092290 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.092259 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:34:09.126367 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.126335 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmwdg\" (UniqueName: \"kubernetes.io/projected/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-kube-api-access-nmwdg\") pod \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " Jul 20 15:34:09.126556 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.126380 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-extensions-socket-volume\") pod \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\" (UID: \"a9136137-9f7c-4c06-a5c8-00d8d071b9d9\") " Jul 20 15:34:09.126733 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.126711 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a9136137-9f7c-4c06-a5c8-00d8d071b9d9" (UID: "a9136137-9f7c-4c06-a5c8-00d8d071b9d9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:34:09.128540 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.128514 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-kube-api-access-nmwdg" (OuterVolumeSpecName: "kube-api-access-nmwdg") pod "a9136137-9f7c-4c06-a5c8-00d8d071b9d9" (UID: "a9136137-9f7c-4c06-a5c8-00d8d071b9d9"). InnerVolumeSpecName "kube-api-access-nmwdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:34:09.227243 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.227199 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmwdg\" (UniqueName: \"kubernetes.io/projected/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-kube-api-access-nmwdg\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:09.227243 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.227244 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9136137-9f7c-4c06-a5c8-00d8d071b9d9-extensions-socket-volume\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:09.254151 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.254107 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8"] Jul 20 15:34:09.254584 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.254565 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9136137-9f7c-4c06-a5c8-00d8d071b9d9" containerName="manager" Jul 20 15:34:09.254664 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.254587 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9136137-9f7c-4c06-a5c8-00d8d071b9d9" containerName="manager" Jul 20 15:34:09.254723 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.254700 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9136137-9f7c-4c06-a5c8-00d8d071b9d9" containerName="manager" Jul 20 15:34:09.258912 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.258892 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.280676 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.280650 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8"] Jul 20 15:34:09.328128 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.328087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ntct8\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.328317 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.328157 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r6kn\" (UniqueName: \"kubernetes.io/projected/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-kube-api-access-5r6kn\") pod \"kuadrant-operator-controller-manager-86878c769f-ntct8\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.429014 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.428959 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r6kn\" (UniqueName: \"kubernetes.io/projected/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-kube-api-access-5r6kn\") pod \"kuadrant-operator-controller-manager-86878c769f-ntct8\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.429414 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.429092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ntct8\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.429507 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.429488 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ntct8\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.439733 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.439706 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r6kn\" (UniqueName: \"kubernetes.io/projected/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-kube-api-access-5r6kn\") pod \"kuadrant-operator-controller-manager-86878c769f-ntct8\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.569726 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.569642 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.706723 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.706693 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8"] Jul 20 15:34:09.708834 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:34:09.708805 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ef952c_ae92_4f23_8fc7_a0e920ed1ae5.slice/crio-7eeefc44e54ad95b02edd7b8aaf708b5d1567959e8850018ed1a3546a26ac6b9 WatchSource:0}: Error finding container 7eeefc44e54ad95b02edd7b8aaf708b5d1567959e8850018ed1a3546a26ac6b9: Status 404 returned error can't find the container with id 7eeefc44e54ad95b02edd7b8aaf708b5d1567959e8850018ed1a3546a26ac6b9 Jul 20 15:34:09.825149 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.825059 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9136137-9f7c-4c06-a5c8-00d8d071b9d9" containerID="476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af" exitCode=0 Jul 20 15:34:09.825149 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.825136 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" Jul 20 15:34:09.825370 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.825144 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" event={"ID":"a9136137-9f7c-4c06-a5c8-00d8d071b9d9","Type":"ContainerDied","Data":"476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af"} Jul 20 15:34:09.825370 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.825202 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc" event={"ID":"a9136137-9f7c-4c06-a5c8-00d8d071b9d9","Type":"ContainerDied","Data":"5b0afda280f79e6e1029c9584d2079753e4f5516dec78497debecf1b32c3590f"} Jul 20 15:34:09.825370 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.825234 2557 scope.go:117] "RemoveContainer" containerID="476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af" Jul 20 15:34:09.827094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.827059 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" event={"ID":"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5","Type":"ContainerStarted","Data":"ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009"} Jul 20 15:34:09.827216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.827120 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" event={"ID":"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5","Type":"ContainerStarted","Data":"7eeefc44e54ad95b02edd7b8aaf708b5d1567959e8850018ed1a3546a26ac6b9"} Jul 20 15:34:09.827216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.827166 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:09.835752 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.835735 2557 scope.go:117] "RemoveContainer" containerID="476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af" Jul 20 15:34:09.836029 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:34:09.836011 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af\": container with ID starting with 476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af not found: ID does not exist" containerID="476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af" Jul 20 15:34:09.836094 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.836042 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af"} err="failed to get container status \"476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af\": rpc error: code = NotFound desc = could not find container \"476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af\": container with ID starting with 476623b430f43d80c8af26d0cfe7b644b229292b99d24a2c2ebf2112c61162af not found: ID does not exist" Jul 20 15:34:09.854129 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.854084 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" podStartSLOduration=0.854071987 podStartE2EDuration="854.071987ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:34:09.851836531 +0000 UTC m=+441.796429900" watchObservedRunningTime="2026-07-20 15:34:09.854071987 +0000 UTC m=+441.798665356" Jul 20 15:34:09.868116 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.868093 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc"] Jul 20 15:34:09.878543 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:09.878518 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xl9wc"] Jul 20 15:34:10.566228 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:10.566201 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9136137-9f7c-4c06-a5c8-00d8d071b9d9" path="/var/lib/kubelet/pods/a9136137-9f7c-4c06-a5c8-00d8d071b9d9/volumes" Jul 20 15:34:20.837165 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:20.837134 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:34:20.895570 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:20.895540 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m"] Jul 20 15:34:20.895759 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:20.895741 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" podUID="3e36e1b3-af13-4e0a-958e-b90b85164881" containerName="manager" containerID="cri-o://9bf59ecb69e8374e4a90fcb5b44fdeb5bffe8e5e68efdb04b8a53fb796c6dfc5" gracePeriod=10 Jul 20 15:34:21.899802 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:21.899749 2557 generic.go:356] "Generic (PLEG): container finished" podID="3e36e1b3-af13-4e0a-958e-b90b85164881" containerID="9bf59ecb69e8374e4a90fcb5b44fdeb5bffe8e5e68efdb04b8a53fb796c6dfc5" exitCode=0 Jul 20 15:34:21.900176 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:21.899828 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" event={"ID":"3e36e1b3-af13-4e0a-958e-b90b85164881","Type":"ContainerDied","Data":"9bf59ecb69e8374e4a90fcb5b44fdeb5bffe8e5e68efdb04b8a53fb796c6dfc5"} Jul 20 15:34:21.900176 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:21.899892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" event={"ID":"3e36e1b3-af13-4e0a-958e-b90b85164881","Type":"ContainerDied","Data":"62a58cba358ebee4bcb68cb071bce2b34fe15aedeab0a4bb32d95fed3b11b68c"} Jul 20 15:34:21.900176 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:21.899915 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62a58cba358ebee4bcb68cb071bce2b34fe15aedeab0a4bb32d95fed3b11b68c" Jul 20 15:34:22.064039 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.063966 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:34:22.142736 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.142706 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckxh8\" (UniqueName: \"kubernetes.io/projected/3e36e1b3-af13-4e0a-958e-b90b85164881-kube-api-access-ckxh8\") pod \"3e36e1b3-af13-4e0a-958e-b90b85164881\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " Jul 20 15:34:22.142903 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.142799 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e36e1b3-af13-4e0a-958e-b90b85164881-extensions-socket-volume\") pod \"3e36e1b3-af13-4e0a-958e-b90b85164881\" (UID: \"3e36e1b3-af13-4e0a-958e-b90b85164881\") " Jul 20 15:34:22.143188 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.143161 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e36e1b3-af13-4e0a-958e-b90b85164881-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3e36e1b3-af13-4e0a-958e-b90b85164881" (UID: "3e36e1b3-af13-4e0a-958e-b90b85164881"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:34:22.145035 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.145010 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e36e1b3-af13-4e0a-958e-b90b85164881-kube-api-access-ckxh8" (OuterVolumeSpecName: "kube-api-access-ckxh8") pod "3e36e1b3-af13-4e0a-958e-b90b85164881" (UID: "3e36e1b3-af13-4e0a-958e-b90b85164881"). InnerVolumeSpecName "kube-api-access-ckxh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:34:22.243648 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.243619 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckxh8\" (UniqueName: \"kubernetes.io/projected/3e36e1b3-af13-4e0a-958e-b90b85164881-kube-api-access-ckxh8\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:22.243648 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.243646 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e36e1b3-af13-4e0a-958e-b90b85164881-extensions-socket-volume\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:22.905269 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.905241 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m" Jul 20 15:34:22.930587 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.930554 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m"] Jul 20 15:34:22.936529 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:22.936502 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d484m"] Jul 20 15:34:24.565681 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:24.565646 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e36e1b3-af13-4e0a-958e-b90b85164881" path="/var/lib/kubelet/pods/3e36e1b3-af13-4e0a-958e-b90b85164881/volumes" Jul 20 15:34:25.371870 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.371841 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b"] Jul 20 15:34:25.372210 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.372198 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e36e1b3-af13-4e0a-958e-b90b85164881" containerName="manager" Jul 20 15:34:25.372262 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.372212 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e36e1b3-af13-4e0a-958e-b90b85164881" containerName="manager" Jul 20 15:34:25.372315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.372279 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e36e1b3-af13-4e0a-958e-b90b85164881" containerName="manager" Jul 20 15:34:25.655962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.655924 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b"] Jul 20 15:34:25.655962 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.655955 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.658614 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.658591 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rzj8n\"" Jul 20 15:34:25.775392 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775576 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775405 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775576 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775431 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775576 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775491 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775576 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775525 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775576 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775578 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775615 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775649 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cq74\" (UniqueName: \"kubernetes.io/projected/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-kube-api-access-7cq74\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.775911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.775708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876613 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876584 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cq74\" (UniqueName: \"kubernetes.io/projected/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-kube-api-access-7cq74\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876613 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876649 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876673 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876699 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876747 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.876886 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.876862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.877233 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.877162 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.877309 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.877276 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.877309 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.877291 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.877451 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.877432 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.877542 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.877527 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.879334 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.879305 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.879474 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.879458 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.885467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.885446 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cq74\" (UniqueName: \"kubernetes.io/projected/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-kube-api-access-7cq74\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.885604 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.885587 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e31d0278-dbd4-491a-a7b5-10a1e7c53d3c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xqn2b\" (UID: \"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:25.966474 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:25.966399 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:26.202034 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.202002 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b"] Jul 20 15:34:26.203181 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:34:26.203151 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31d0278_dbd4_491a_a7b5_10a1e7c53d3c.slice/crio-a4e888d93543ee4d6f2a14fe60f9ffae20036c4e2162802930a30ecd0138dee1 WatchSource:0}: Error finding container a4e888d93543ee4d6f2a14fe60f9ffae20036c4e2162802930a30ecd0138dee1: Status 404 returned error can't find the container with id a4e888d93543ee4d6f2a14fe60f9ffae20036c4e2162802930a30ecd0138dee1 Jul 20 15:34:26.205440 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.205405 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:34:26.205520 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.205471 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:34:26.205520 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.205498 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:34:26.929501 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.929465 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" event={"ID":"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c","Type":"ContainerStarted","Data":"ed004dc4fa2ce0d6abf0d302d47224b3d43071e36486c0afb08fdddbebc37619"} Jul 20 15:34:26.929864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.929530 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" event={"ID":"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c","Type":"ContainerStarted","Data":"a4e888d93543ee4d6f2a14fe60f9ffae20036c4e2162802930a30ecd0138dee1"} Jul 20 15:34:26.950358 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.950313 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" podStartSLOduration=1.950298404 podStartE2EDuration="1.950298404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:34:26.947688081 +0000 UTC m=+458.892281451" watchObservedRunningTime="2026-07-20 15:34:26.950298404 +0000 UTC m=+458.894891774" Jul 20 15:34:26.966980 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:26.966942 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:27.972132 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:27.972106 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:28.941226 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:28.941196 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:28.942159 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:28.942141 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:34:33.908639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:33.908558 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b8959f7c6-47kcb" podUID="0dc2449b-3b52-41a2-a8e5-e9970ba1794f" containerName="console" containerID="cri-o://e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab" gracePeriod=15 Jul 20 15:34:34.155704 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.155682 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:34:34.244573 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244493 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njbl8\" (UniqueName: \"kubernetes.io/projected/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-kube-api-access-njbl8\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.244573 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244532 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-oauth-config\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.244573 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244555 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-trusted-ca-bundle\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.244573 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244570 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-oauth-serving-cert\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.244891 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244639 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-serving-cert\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.244891 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244697 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-config\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.244891 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.244752 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-service-ca\") pod \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\" (UID: \"0dc2449b-3b52-41a2-a8e5-e9970ba1794f\") " Jul 20 15:34:34.245132 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.245103 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:34:34.245293 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.245117 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:34:34.245392 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.245329 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-service-ca" (OuterVolumeSpecName: "service-ca") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:34:34.245447 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.245427 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-config" (OuterVolumeSpecName: "console-config") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:34:34.246938 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.246915 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:34:34.246938 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.246925 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-kube-api-access-njbl8" (OuterVolumeSpecName: "kube-api-access-njbl8") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "kube-api-access-njbl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:34:34.247025 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.246930 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0dc2449b-3b52-41a2-a8e5-e9970ba1794f" (UID: "0dc2449b-3b52-41a2-a8e5-e9970ba1794f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 15:34:34.345453 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345423 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.345453 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345448 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.345453 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345458 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-service-ca\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.345641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345467 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njbl8\" (UniqueName: \"kubernetes.io/projected/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-kube-api-access-njbl8\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.345641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345476 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-console-oauth-config\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.345641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345484 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-trusted-ca-bundle\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.345641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.345492 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dc2449b-3b52-41a2-a8e5-e9970ba1794f-oauth-serving-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:34:34.976585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.976551 2557 generic.go:356] "Generic (PLEG): container finished" podID="0dc2449b-3b52-41a2-a8e5-e9970ba1794f" containerID="e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab" exitCode=0 Jul 20 15:34:34.976991 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.976599 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b8959f7c6-47kcb" Jul 20 15:34:34.976991 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.976645 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b8959f7c6-47kcb" event={"ID":"0dc2449b-3b52-41a2-a8e5-e9970ba1794f","Type":"ContainerDied","Data":"e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab"} Jul 20 15:34:34.976991 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.976703 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b8959f7c6-47kcb" event={"ID":"0dc2449b-3b52-41a2-a8e5-e9970ba1794f","Type":"ContainerDied","Data":"2c1649c832cd849e6b1e455f27085dc6984993cec1db37d8e4faf9c3527c160d"} Jul 20 15:34:34.976991 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.976723 2557 scope.go:117] "RemoveContainer" containerID="e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab" Jul 20 15:34:34.985824 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.985803 2557 scope.go:117] "RemoveContainer" containerID="e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab" Jul 20 15:34:34.986113 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:34:34.986079 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab\": container with ID starting with e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab not found: ID does not exist" containerID="e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab" Jul 20 15:34:34.986208 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:34.986110 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab"} err="failed to get container status \"e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab\": rpc error: code = NotFound desc = could not find container \"e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab\": container with ID starting with e76f1b1527ae335ba5bff0adb86a6225d2965bce3f50ec93933724c097b229ab not found: ID does not exist" Jul 20 15:34:35.004340 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:35.004316 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b8959f7c6-47kcb"] Jul 20 15:34:35.011473 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:35.011451 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b8959f7c6-47kcb"] Jul 20 15:34:36.566455 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:36.566424 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dc2449b-3b52-41a2-a8e5-e9970ba1794f" path="/var/lib/kubelet/pods/0dc2449b-3b52-41a2-a8e5-e9970ba1794f/volumes" Jul 20 15:34:42.849113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.849068 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:34:42.849545 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.849453 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc2449b-3b52-41a2-a8e5-e9970ba1794f" containerName="console" Jul 20 15:34:42.849545 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.849469 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc2449b-3b52-41a2-a8e5-e9970ba1794f" containerName="console" Jul 20 15:34:42.849621 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.849549 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dc2449b-3b52-41a2-a8e5-e9970ba1794f" containerName="console" Jul 20 15:34:42.851658 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.851638 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:42.853638 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.853616 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 15:34:42.861638 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.861612 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:34:42.890866 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.890818 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:34:42.914164 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.914127 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tpcm\" (UniqueName: \"kubernetes.io/projected/97641873-5e39-44c8-ab98-211a181c8d49-kube-api-access-7tpcm\") pod \"limitador-limitador-54d6d988d5-br5t2\" (UID: \"97641873-5e39-44c8-ab98-211a181c8d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:42.914323 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:42.914173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97641873-5e39-44c8-ab98-211a181c8d49-config-file\") pod \"limitador-limitador-54d6d988d5-br5t2\" (UID: \"97641873-5e39-44c8-ab98-211a181c8d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:43.015463 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:43.015429 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tpcm\" (UniqueName: \"kubernetes.io/projected/97641873-5e39-44c8-ab98-211a181c8d49-kube-api-access-7tpcm\") pod \"limitador-limitador-54d6d988d5-br5t2\" (UID: \"97641873-5e39-44c8-ab98-211a181c8d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:43.015637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:43.015474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97641873-5e39-44c8-ab98-211a181c8d49-config-file\") pod \"limitador-limitador-54d6d988d5-br5t2\" (UID: \"97641873-5e39-44c8-ab98-211a181c8d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:43.016185 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:43.016161 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/97641873-5e39-44c8-ab98-211a181c8d49-config-file\") pod \"limitador-limitador-54d6d988d5-br5t2\" (UID: \"97641873-5e39-44c8-ab98-211a181c8d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:43.023395 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:43.023373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tpcm\" (UniqueName: \"kubernetes.io/projected/97641873-5e39-44c8-ab98-211a181c8d49-kube-api-access-7tpcm\") pod \"limitador-limitador-54d6d988d5-br5t2\" (UID: \"97641873-5e39-44c8-ab98-211a181c8d49\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:43.162639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:43.162605 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:43.305474 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:43.305448 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:34:44.027793 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:44.027729 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" event={"ID":"97641873-5e39-44c8-ab98-211a181c8d49","Type":"ContainerStarted","Data":"c467a5f718e0d6359563e579a427b0969402089da7e182cdcee0fa3bb3c0868b"} Jul 20 15:34:47.049663 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:47.049629 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" event={"ID":"97641873-5e39-44c8-ab98-211a181c8d49","Type":"ContainerStarted","Data":"5b9681bb586d56cf7d7b7a28c59fd03b8bf4be2623358d9733850f165055eba5"} Jul 20 15:34:47.050091 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:47.049796 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:34:47.067159 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:47.067118 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" podStartSLOduration=2.003449919 podStartE2EDuration="5.067100511s" totalImagesPullingTime="3.063650592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:34:47.065337181 +0000 UTC m=+479.009930552" watchObservedRunningTime="2026-07-20 15:34:47.067100511 +0000 UTC m=+479.011693881" Jul 20 15:34:58.055902 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:34:58.055865 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-br5t2" Jul 20 15:35:58.627221 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:35:58.627188 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:36:06.490192 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:06.490157 2557 generic.go:356] "Generic (PLEG): container finished" podID="e31d0278-dbd4-491a-a7b5-10a1e7c53d3c" containerID="ed004dc4fa2ce0d6abf0d302d47224b3d43071e36486c0afb08fdddbebc37619" exitCode=0 Jul 20 15:36:06.490711 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:06.490235 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" event={"ID":"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c","Type":"ContainerDied","Data":"ed004dc4fa2ce0d6abf0d302d47224b3d43071e36486c0afb08fdddbebc37619"} Jul 20 15:36:06.490805 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:06.490724 2557 scope.go:117] "RemoveContainer" containerID="ed004dc4fa2ce0d6abf0d302d47224b3d43071e36486c0afb08fdddbebc37619" Jul 20 15:36:06.491759 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:06.491733 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:36:06.491850 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:06.491816 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:36:06.491890 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:06.491849 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:36:07.499216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:07.499184 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" event={"ID":"e31d0278-dbd4-491a-a7b5-10a1e7c53d3c","Type":"ContainerStarted","Data":"9b91ac054b98866b1f2222189ef79c8b8114136fe39983dd08b4c30186d5cb99"} Jul 20 15:36:07.967251 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:07.967212 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:36:07.967433 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:07.967400 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xqn2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 15:36:07.967482 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:07.967447 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" podUID="e31d0278-dbd4-491a-a7b5-10a1e7c53d3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 15:36:08.967782 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:08.967744 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xqn2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 15:36:08.968201 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:08.967830 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" podUID="e31d0278-dbd4-491a-a7b5-10a1e7c53d3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 15:36:09.967638 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:09.967602 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xqn2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 15:36:09.967837 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:09.967666 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" podUID="e31d0278-dbd4-491a-a7b5-10a1e7c53d3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 15:36:10.967391 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:10.967350 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xqn2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 15:36:10.967571 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:10.967407 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" podUID="e31d0278-dbd4-491a-a7b5-10a1e7c53d3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 15:36:11.970800 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:11.970735 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:36:11.971216 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:11.971047 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:36:11.971698 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:11.971677 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xqn2b" Jul 20 15:36:15.684463 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:15.684423 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:36:31.578722 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:31.578683 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:36:38.591914 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:38.591880 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:36:48.673205 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:48.673171 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:36:48.674176 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:48.674153 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:36:49.884068 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.884037 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2"] Jul 20 15:36:49.887500 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.887480 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:49.890220 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.890199 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 15:36:49.890345 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.890267 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 20 15:36:49.890345 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.890338 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 15:36:49.890475 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.890361 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wqs5r\"" Jul 20 15:36:49.901182 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.901147 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2"] Jul 20 15:36:49.980684 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.980648 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:49.980684 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.980685 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv7gq\" (UniqueName: \"kubernetes.io/projected/3582e79a-91a3-4387-a0a7-3c923ae3135a-kube-api-access-tv7gq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:49.980922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.980744 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3582e79a-91a3-4387-a0a7-3c923ae3135a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:49.980922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.980822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:49.980922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.980863 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:49.980922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:49.980883 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.081639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.081599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.081870 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.081644 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.081870 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.081690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.081870 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.081715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv7gq\" (UniqueName: \"kubernetes.io/projected/3582e79a-91a3-4387-a0a7-3c923ae3135a-kube-api-access-tv7gq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.081870 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.081763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3582e79a-91a3-4387-a0a7-3c923ae3135a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.081870 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.081837 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.082154 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.082058 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.082154 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.082131 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.082263 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.082222 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.084023 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.084001 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3582e79a-91a3-4387-a0a7-3c923ae3135a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.084222 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.084205 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3582e79a-91a3-4387-a0a7-3c923ae3135a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.089006 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.088986 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv7gq\" (UniqueName: \"kubernetes.io/projected/3582e79a-91a3-4387-a0a7-3c923ae3135a-kube-api-access-tv7gq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2\" (UID: \"3582e79a-91a3-4387-a0a7-3c923ae3135a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.197950 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.197860 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:36:50.340179 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.340148 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2"] Jul 20 15:36:50.341884 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:36:50.341859 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3582e79a_91a3_4387_a0a7_3c923ae3135a.slice/crio-9a6a8a1dcb503df7af6b9afd82819de43e97adef3c7c51e6a1419d6f93a30bb4 WatchSource:0}: Error finding container 9a6a8a1dcb503df7af6b9afd82819de43e97adef3c7c51e6a1419d6f93a30bb4: Status 404 returned error can't find the container with id 9a6a8a1dcb503df7af6b9afd82819de43e97adef3c7c51e6a1419d6f93a30bb4 Jul 20 15:36:50.756816 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:50.756749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" event={"ID":"3582e79a-91a3-4387-a0a7-3c923ae3135a","Type":"ContainerStarted","Data":"9a6a8a1dcb503df7af6b9afd82819de43e97adef3c7c51e6a1419d6f93a30bb4"} Jul 20 15:36:51.199461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:51.199419 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:36:51.764323 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:51.764237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" event={"ID":"3582e79a-91a3-4387-a0a7-3c923ae3135a","Type":"ContainerStarted","Data":"d7bd7c84a3c5bf58248673376ce212aff4f58c83e039bbeca3d8e478649f3ab9"} Jul 20 15:36:51.781032 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:36:51.780974 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" podStartSLOduration=1.677670364 podStartE2EDuration="2.780954652s" totalImagesPullingTime="1.103284288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:36:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:36:51.780125552 +0000 UTC m=+603.724718923" watchObservedRunningTime="2026-07-20 15:36:51.780954652 +0000 UTC m=+603.725548081" Jul 20 15:37:00.198627 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:00.198576 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:37:00.198627 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:00.198644 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:37:00.211394 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:00.211366 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:37:00.675199 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:00.675161 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:37:00.823762 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:00.823729 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2" Jul 20 15:37:03.798397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.798297 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz"] Jul 20 15:37:03.802124 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.802102 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:03.804282 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.804256 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 20 15:37:03.812393 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.812372 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz"] Jul 20 15:37:03.907235 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.907193 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:03.907235 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.907240 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:03.907467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.907277 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:03.907467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.907300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:03.907467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.907330 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9skf\" (UniqueName: \"kubernetes.io/projected/a4c35af4-9e30-474b-9090-7d28319f78e0-kube-api-access-d9skf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:03.907467 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:03.907386 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c35af4-9e30-474b-9090-7d28319f78e0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008137 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008102 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c35af4-9e30-474b-9090-7d28319f78e0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008305 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008183 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008305 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008305 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008305 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008275 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008487 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9skf\" (UniqueName: \"kubernetes.io/projected/a4c35af4-9e30-474b-9090-7d28319f78e0-kube-api-access-d9skf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008694 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008830 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008725 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.008830 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.008795 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.010523 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.010505 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c35af4-9e30-474b-9090-7d28319f78e0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.010934 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.010913 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c35af4-9e30-474b-9090-7d28319f78e0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.025726 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.025701 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9skf\" (UniqueName: \"kubernetes.io/projected/a4c35af4-9e30-474b-9090-7d28319f78e0-kube-api-access-d9skf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz\" (UID: \"a4c35af4-9e30-474b-9090-7d28319f78e0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.111464 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.111403 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:04.254501 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.254473 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz"] Jul 20 15:37:04.256426 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:37:04.256395 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c35af4_9e30_474b_9090_7d28319f78e0.slice/crio-d4abfa9ef011461a21f8eb47278f120c0cd0a5c0a8aa79785c9150607007a143 WatchSource:0}: Error finding container d4abfa9ef011461a21f8eb47278f120c0cd0a5c0a8aa79785c9150607007a143: Status 404 returned error can't find the container with id d4abfa9ef011461a21f8eb47278f120c0cd0a5c0a8aa79785c9150607007a143 Jul 20 15:37:04.257883 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.257862 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 15:37:04.838639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.838527 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" event={"ID":"a4c35af4-9e30-474b-9090-7d28319f78e0","Type":"ContainerStarted","Data":"c4214b30a01230ab39338b4dbbba356e7096668e2f12499318e1c10f68e7efb4"} Jul 20 15:37:04.838639 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.838581 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" event={"ID":"a4c35af4-9e30-474b-9090-7d28319f78e0","Type":"ContainerStarted","Data":"d4abfa9ef011461a21f8eb47278f120c0cd0a5c0a8aa79785c9150607007a143"} Jul 20 15:37:04.857361 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:04.857309 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" podStartSLOduration=1.6864206720000001 podStartE2EDuration="1.857292919s" totalImagesPullingTime="170.872247ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:37:04.855286758 +0000 UTC m=+616.799880129" watchObservedRunningTime="2026-07-20 15:37:04.857292919 +0000 UTC m=+616.801886290" Jul 20 15:37:06.688922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:06.688885 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:37:14.111984 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:14.111903 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:14.111984 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:14.111960 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:14.124721 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:14.124699 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:37:14.906990 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:37:14.906959 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz" Jul 20 15:38:34.897301 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:38:34.897262 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:38:45.199472 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:38:45.199437 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:38:56.289394 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:38:56.289358 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:40:37.390829 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:40:37.390707 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:40:48.547431 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:40:48.547401 2557 scope.go:117] "RemoveContainer" containerID="9bf59ecb69e8374e4a90fcb5b44fdeb5bffe8e5e68efdb04b8a53fb796c6dfc5" Jul 20 15:40:50.784179 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:40:50.784149 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:41:24.190693 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:24.190654 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:41:33.876324 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:33.876288 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:41:41.775096 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:41.775012 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:41:48.708857 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:48.708827 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:41:48.712192 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:48.712174 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:41:49.285489 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:49.285457 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:41:58.785690 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:41:58.785656 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:42:08.086797 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:42:08.086737 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:42:19.592190 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:42:19.592155 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:42:29.380034 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:42:29.379999 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:42:52.578645 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:42:52.578612 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:43:01.182207 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:43:01.182164 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:43:25.387350 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:43:25.387272 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:43:35.585855 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:43:35.585817 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:43:54.376097 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:43:54.376064 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:44:04.094585 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:44:04.094546 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:44:15.916188 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:44:15.916144 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:44:25.299542 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:44:25.299502 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:44:34.706465 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:44:34.706435 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:45:00.254664 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:00.254620 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8"] Jul 20 15:45:00.255234 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:00.254888 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" podUID="38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" containerName="manager" containerID="cri-o://ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009" gracePeriod=10 Jul 20 15:45:03.917141 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:03.917115 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:45:04.001315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.001245 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r6kn\" (UniqueName: \"kubernetes.io/projected/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-kube-api-access-5r6kn\") pod \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " Jul 20 15:45:04.001315 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.001283 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-extensions-socket-volume\") pod \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\" (UID: \"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5\") " Jul 20 15:45:04.001665 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.001643 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" (UID: "38ef952c-ae92-4f23-8fc7-a0e920ed1ae5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:45:04.003655 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.003624 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-kube-api-access-5r6kn" (OuterVolumeSpecName: "kube-api-access-5r6kn") pod "38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" (UID: "38ef952c-ae92-4f23-8fc7-a0e920ed1ae5"). InnerVolumeSpecName "kube-api-access-5r6kn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:45:04.102150 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.102121 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r6kn\" (UniqueName: \"kubernetes.io/projected/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-kube-api-access-5r6kn\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:45:04.102150 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.102146 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5-extensions-socket-volume\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:45:04.562401 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.562369 2557 generic.go:356] "Generic (PLEG): container finished" podID="38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" containerID="ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009" exitCode=0 Jul 20 15:45:04.562566 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.562458 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" Jul 20 15:45:04.566628 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.566603 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" event={"ID":"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5","Type":"ContainerDied","Data":"ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009"} Jul 20 15:45:04.566737 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.566653 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8" event={"ID":"38ef952c-ae92-4f23-8fc7-a0e920ed1ae5","Type":"ContainerDied","Data":"7eeefc44e54ad95b02edd7b8aaf708b5d1567959e8850018ed1a3546a26ac6b9"} Jul 20 15:45:04.566737 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.566686 2557 scope.go:117] "RemoveContainer" containerID="ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009" Jul 20 15:45:04.577725 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.577708 2557 scope.go:117] "RemoveContainer" containerID="ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009" Jul 20 15:45:04.578004 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:45:04.577987 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009\": container with ID starting with ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009 not found: ID does not exist" containerID="ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009" Jul 20 15:45:04.578078 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.578013 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009"} err="failed to get container status \"ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009\": rpc error: code = NotFound desc = could not find container \"ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009\": container with ID starting with ad121d0797bafa1d200664e6ec62de431a922413b3a4648ab68478361a3e9009 not found: ID does not exist" Jul 20 15:45:04.588559 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.588538 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8"] Jul 20 15:45:04.593383 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:04.593361 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ntct8"] Jul 20 15:45:06.571666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:45:06.571633 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" path="/var/lib/kubelet/pods/38ef952c-ae92-4f23-8fc7-a0e920ed1ae5/volumes" Jul 20 15:46:01.480250 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.480218 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd"] Jul 20 15:46:01.480819 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.480801 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" containerName="manager" Jul 20 15:46:01.480913 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.480822 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" containerName="manager" Jul 20 15:46:01.480975 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.480956 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="38ef952c-ae92-4f23-8fc7-a0e920ed1ae5" containerName="manager" Jul 20 15:46:01.485267 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.485244 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.488591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.488574 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zdfnq\"" Jul 20 15:46:01.500596 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.500564 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd"] Jul 20 15:46:01.591366 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.591335 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dd01d83-693f-47e1-a193-8d50af239cda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x95zd\" (UID: \"4dd01d83-693f-47e1-a193-8d50af239cda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.591541 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.591394 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mscw\" (UniqueName: \"kubernetes.io/projected/4dd01d83-693f-47e1-a193-8d50af239cda-kube-api-access-8mscw\") pod \"kuadrant-operator-controller-manager-86878c769f-x95zd\" (UID: \"4dd01d83-693f-47e1-a193-8d50af239cda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.692138 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.692108 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dd01d83-693f-47e1-a193-8d50af239cda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x95zd\" (UID: \"4dd01d83-693f-47e1-a193-8d50af239cda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.692299 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.692149 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mscw\" (UniqueName: \"kubernetes.io/projected/4dd01d83-693f-47e1-a193-8d50af239cda-kube-api-access-8mscw\") pod \"kuadrant-operator-controller-manager-86878c769f-x95zd\" (UID: \"4dd01d83-693f-47e1-a193-8d50af239cda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.692568 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.692543 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4dd01d83-693f-47e1-a193-8d50af239cda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x95zd\" (UID: \"4dd01d83-693f-47e1-a193-8d50af239cda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.700603 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.700581 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mscw\" (UniqueName: \"kubernetes.io/projected/4dd01d83-693f-47e1-a193-8d50af239cda-kube-api-access-8mscw\") pod \"kuadrant-operator-controller-manager-86878c769f-x95zd\" (UID: \"4dd01d83-693f-47e1-a193-8d50af239cda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.795812 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.795688 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:01.944234 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:01.944205 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd"] Jul 20 15:46:01.946410 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:46:01.946385 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd01d83_693f_47e1_a193_8d50af239cda.slice/crio-c5e79181ba570d4795ccd356c59a46ffb6161c00751499b63a9c4c00a2f808c7 WatchSource:0}: Error finding container c5e79181ba570d4795ccd356c59a46ffb6161c00751499b63a9c4c00a2f808c7: Status 404 returned error can't find the container with id c5e79181ba570d4795ccd356c59a46ffb6161c00751499b63a9c4c00a2f808c7 Jul 20 15:46:02.878280 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:02.878245 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" event={"ID":"4dd01d83-693f-47e1-a193-8d50af239cda","Type":"ContainerStarted","Data":"e23ed228c5373a5f9dd1d70e4a85b2561ed5657633b1d0211745b495a3009f8c"} Jul 20 15:46:02.878699 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:02.878298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" event={"ID":"4dd01d83-693f-47e1-a193-8d50af239cda","Type":"ContainerStarted","Data":"c5e79181ba570d4795ccd356c59a46ffb6161c00751499b63a9c4c00a2f808c7"} Jul 20 15:46:02.878699 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:02.878353 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:02.898616 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:02.898566 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" podStartSLOduration=1.898552115 podStartE2EDuration="1.898552115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:46:02.896623296 +0000 UTC m=+1154.841216667" watchObservedRunningTime="2026-07-20 15:46:02.898552115 +0000 UTC m=+1154.843145485" Jul 20 15:46:13.886454 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:13.886374 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x95zd" Jul 20 15:46:24.181174 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:24.181137 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:46:35.191720 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:35.185830 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:46:43.984641 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:43.984605 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:46:48.747157 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:48.747131 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:46:48.750803 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:48.750758 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:46:56.881887 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:46:56.881854 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:06.080392 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:06.080353 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:15.481854 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:15.481813 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:23.977072 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:23.977032 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:32.981858 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:32.981820 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:40.575715 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:40.575624 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:51.481158 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:51.481121 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:47:59.478966 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:47:59.478926 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:48:08.278422 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:48:08.278389 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:48:32.083153 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:48:32.083115 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:48:41.483470 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:48:41.483435 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:48:57.188222 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:48:57.188177 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:49:11.883856 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:49:11.883752 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:49:22.183841 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:49:22.183803 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:49:35.086482 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:49:35.086447 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:49:41.677033 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:49:41.676998 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:49:50.582141 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:49:50.582103 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:49:58.384249 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:49:58.384214 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:50:08.475323 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:50:08.475291 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:50:17.886141 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:50:17.886102 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:50:28.274831 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:50:28.274791 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:50:37.296692 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:50:37.296650 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:50:46.691386 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:50:46.691303 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:50:59.591976 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:50:59.591940 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:51:08.586494 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:08.586458 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:51:18.294539 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:18.294497 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:51:30.578101 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:30.578069 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:51:39.984199 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:39.984167 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:51:46.609878 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:46.609849 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:51:48.789005 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:48.788979 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:51:48.792569 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:51:48.792549 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:52:01.098713 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:01.098674 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:52:11.475234 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:11.475152 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:52:22.798926 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:22.798889 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:52:34.278221 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.278185 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg"] Jul 20 15:52:34.282059 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.282034 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.284305 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.284286 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a1d4549a-openshift-default-dockercfg-kbg96\"" Jul 20 15:52:34.303837 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.303802 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg"] Jul 20 15:52:34.446754 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.446715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-credential-socket\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.446948 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.446765 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-socket\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.446948 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.446852 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-data\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.446948 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.446877 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-token\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.446948 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.446904 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-podinfo\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.447087 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.446974 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-envoy\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.447087 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.447018 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9x75\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-kube-api-access-x9x75\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.447087 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.447042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-certs\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.447087 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.447064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5673c1-9c29-4126-b495-c094bc4caf4a-istiod-ca-cert\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.547821 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.547707 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-socket\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.547821 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.547749 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-data\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548014 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.547946 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-token\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548014 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.547973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-podinfo\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-envoy\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9x75\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-kube-api-access-x9x75\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-certs\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5673c1-9c29-4126-b495-c094bc4caf4a-istiod-ca-cert\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-socket\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548178 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548157 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-data\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548473 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548261 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-credential-socket\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548520 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548503 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-certs\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548577 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548555 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-credential-socket\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.548790 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.548757 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5673c1-9c29-4126-b495-c094bc4caf4a-istiod-ca-cert\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.550397 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.550377 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-envoy\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.550621 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.550603 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-podinfo\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.556226 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.556197 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-token\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.556348 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.556301 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9x75\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-kube-api-access-x9x75\") pod \"e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.592328 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.592294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:34.743593 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.743559 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg"] Jul 20 15:52:34.744963 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:52:34.744934 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a5673c1_9c29_4126_b495_c094bc4caf4a.slice/crio-66793dd76097e3885626f975788b98ff787e076239e97d1f7763e3b02554619b WatchSource:0}: Error finding container 66793dd76097e3885626f975788b98ff787e076239e97d1f7763e3b02554619b: Status 404 returned error can't find the container with id 66793dd76097e3885626f975788b98ff787e076239e97d1f7763e3b02554619b Jul 20 15:52:34.747285 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.747245 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:52:34.747373 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.747331 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:52:34.747412 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:34.747373 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:52:35.118922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:35.118828 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" event={"ID":"2a5673c1-9c29-4126-b495-c094bc4caf4a","Type":"ContainerStarted","Data":"88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415"} Jul 20 15:52:35.118922 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:35.118893 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" event={"ID":"2a5673c1-9c29-4126-b495-c094bc4caf4a","Type":"ContainerStarted","Data":"66793dd76097e3885626f975788b98ff787e076239e97d1f7763e3b02554619b"} Jul 20 15:52:35.138629 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:35.138574 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podStartSLOduration=1.138559134 podStartE2EDuration="1.138559134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:52:35.136479605 +0000 UTC m=+1547.081072975" watchObservedRunningTime="2026-07-20 15:52:35.138559134 +0000 UTC m=+1547.083152503" Jul 20 15:52:35.592892 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:35.592851 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:35.597713 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:35.597686 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:36.124331 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:36.124301 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:36.125234 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:36.125212 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:44.296758 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:44.296719 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:52:52.165969 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:52.165921 2557 patch_prober.go:28] interesting pod/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 15:52:52.166377 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:52.166006 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 15:52:53.222706 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:53.222665 2557 generic.go:356] "Generic (PLEG): container finished" podID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerID="88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415" exitCode=0 Jul 20 15:52:53.223272 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:53.222738 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" event={"ID":"2a5673c1-9c29-4126-b495-c094bc4caf4a","Type":"ContainerDied","Data":"88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415"} Jul 20 15:52:53.223272 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:53.223223 2557 scope.go:117] "RemoveContainer" containerID="88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415" Jul 20 15:52:53.224256 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:53.224231 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:52:53.224337 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:53.224320 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:52:53.224394 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:53.224352 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:52:54.235498 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:54.235436 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" event={"ID":"2a5673c1-9c29-4126-b495-c094bc4caf4a","Type":"ContainerStarted","Data":"63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e"} Jul 20 15:52:54.592748 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:54.592658 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:54.592952 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:54.592931 2557 patch_prober.go:28] interesting pod/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 15:52:54.592997 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:54.592978 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 15:52:55.593655 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:55.593618 2557 patch_prober.go:28] interesting pod/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 15:52:55.594081 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:55.593670 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 15:52:56.592830 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:56.592765 2557 patch_prober.go:28] interesting pod/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 15:52:56.593008 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:56.592844 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 15:52:57.593793 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:57.593723 2557 patch_prober.go:28] interesting pod/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 15:52:57.594189 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:57.593823 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 15:52:58.596106 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:58.596074 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:58.596480 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:58.596334 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:52:58.597039 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:52:58.597018 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:53:15.696404 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:15.696366 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:53:17.388933 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:17.388890 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg"] Jul 20 15:53:17.389405 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:17.389100 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" containerID="cri-o://63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e" gracePeriod=30 Jul 20 15:53:22.647502 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.647477 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:53:22.702385 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702315 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-credential-socket\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702385 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702371 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-podinfo\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702396 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9x75\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-kube-api-access-x9x75\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702420 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-socket\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702458 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-certs\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702546 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-token\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702591 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702589 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-envoy\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702882 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702607 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:22.702882 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702618 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5673c1-9c29-4126-b495-c094bc4caf4a-istiod-ca-cert\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702882 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702695 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-data\") pod \"2a5673c1-9c29-4126-b495-c094bc4caf4a\" (UID: \"2a5673c1-9c29-4126-b495-c094bc4caf4a\") " Jul 20 15:53:22.702882 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702721 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:22.702882 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.702762 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:22.703144 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.703021 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.703144 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.703042 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-workload-certs\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.703144 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.703059 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-credential-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.703144 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.703059 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a5673c1-9c29-4126-b495-c094bc4caf4a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:53:22.703596 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.703575 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:22.704838 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.704814 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:53:22.704960 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.704848 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:53:22.705310 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.705291 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 15:53:22.705310 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.705294 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-kube-api-access-x9x75" (OuterVolumeSpecName: "kube-api-access-x9x75") pod "2a5673c1-9c29-4126-b495-c094bc4caf4a" (UID: "2a5673c1-9c29-4126-b495-c094bc4caf4a"). InnerVolumeSpecName "kube-api-access-x9x75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:53:22.804475 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.804443 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-data\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.804475 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.804470 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-podinfo\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.804475 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.804480 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9x75\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-kube-api-access-x9x75\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.804694 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.804489 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-token\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.804694 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.804498 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a5673c1-9c29-4126-b495-c094bc4caf4a-istio-envoy\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:22.804694 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:22.804506 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a5673c1-9c29-4126-b495-c094bc4caf4a-istiod-ca-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:53:23.414707 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.414674 2557 generic.go:356] "Generic (PLEG): container finished" podID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerID="63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e" exitCode=0 Jul 20 15:53:23.414916 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.414721 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" Jul 20 15:53:23.414916 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.414760 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" event={"ID":"2a5673c1-9c29-4126-b495-c094bc4caf4a","Type":"ContainerDied","Data":"63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e"} Jul 20 15:53:23.414916 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.414842 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg" event={"ID":"2a5673c1-9c29-4126-b495-c094bc4caf4a","Type":"ContainerDied","Data":"66793dd76097e3885626f975788b98ff787e076239e97d1f7763e3b02554619b"} Jul 20 15:53:23.414916 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.414871 2557 scope.go:117] "RemoveContainer" containerID="63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e" Jul 20 15:53:23.426260 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.426241 2557 scope.go:117] "RemoveContainer" containerID="88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415" Jul 20 15:53:23.434943 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.434910 2557 scope.go:117] "RemoveContainer" containerID="63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e" Jul 20 15:53:23.435232 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:53:23.435210 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e\": container with ID starting with 63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e not found: ID does not exist" containerID="63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e" Jul 20 15:53:23.435361 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.435273 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e"} err="failed to get container status \"63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e\": rpc error: code = NotFound desc = could not find container \"63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e\": container with ID starting with 63cc6a078a6e8d9c35cb38c52ef1da69272ac0c2fb024ead369f2a275ba97e6e not found: ID does not exist" Jul 20 15:53:23.435361 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.435314 2557 scope.go:117] "RemoveContainer" containerID="88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415" Jul 20 15:53:23.435644 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:53:23.435611 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415\": container with ID starting with 88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415 not found: ID does not exist" containerID="88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415" Jul 20 15:53:23.435763 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.435652 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415"} err="failed to get container status \"88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415\": rpc error: code = NotFound desc = could not find container \"88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415\": container with ID starting with 88964558b0267ccda845827f6171e3c3c9c99f22574d51810d8f841c17a26415 not found: ID does not exist" Jul 20 15:53:23.437828 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.437799 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg"] Jul 20 15:53:23.443184 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:23.443164 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a1d4549a-openshift-default-54d87f599c-bvggg"] Jul 20 15:53:24.566322 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:24.566293 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" path="/var/lib/kubelet/pods/2a5673c1-9c29-4126-b495-c094bc4caf4a/volumes" Jul 20 15:53:27.489474 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:53:27.489436 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:54:03.277192 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:03.277156 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:54:05.834632 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.834601 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6"] Jul 20 15:54:05.835110 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.835093 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" Jul 20 15:54:05.835158 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.835117 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" Jul 20 15:54:05.835271 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.835258 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" Jul 20 15:54:05.835311 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.835277 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" Jul 20 15:54:05.839897 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.839876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.841812 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.841790 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dde17afb-openshift-default-dockercfg-rs5hj\"" Jul 20 15:54:05.851560 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.851536 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6"] Jul 20 15:54:05.902906 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.902870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-podinfo\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.902912 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istiod-ca-cert\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.902933 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-certs\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.902953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-envoy\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903085 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.903055 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-socket\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903317 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.903117 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-credential-socket\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903317 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.903155 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-token\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903317 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.903186 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtc6h\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-kube-api-access-jtc6h\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:05.903317 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:05.903235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-data\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004169 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004133 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-token\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004372 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtc6h\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-kube-api-access-jtc6h\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004372 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004304 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-data\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004372 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004367 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-podinfo\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004554 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istiod-ca-cert\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004554 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-certs\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-envoy\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004637 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-socket\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004742 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-credential-socket\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004819 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004748 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-data\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.004899 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004879 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-certs\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.005007 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.004984 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-socket\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.005048 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.005034 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-credential-socket\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.005114 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.005098 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istiod-ca-cert\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.006913 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.006889 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-envoy\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.007041 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.007024 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-podinfo\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.011466 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.011444 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-token\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.011550 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.011495 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtc6h\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-kube-api-access-jtc6h\") pod \"e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.151204 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.151170 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:06.300174 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.300142 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6"] Jul 20 15:54:06.300866 ip-10-0-141-150 kubenswrapper[2557]: W0720 15:54:06.300838 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5499e52_703f_4bb4_9ea8_1a3772c0d25a.slice/crio-882448d067e80df704d01d7b2bcea3174516f4b0590c3bbc8ac61bc842e9f031 WatchSource:0}: Error finding container 882448d067e80df704d01d7b2bcea3174516f4b0590c3bbc8ac61bc842e9f031: Status 404 returned error can't find the container with id 882448d067e80df704d01d7b2bcea3174516f4b0590c3bbc8ac61bc842e9f031 Jul 20 15:54:06.303054 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.303018 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:54:06.303142 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.303083 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:54:06.303142 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.303110 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:54:06.658273 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.658240 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" event={"ID":"d5499e52-703f-4bb4-9ea8-1a3772c0d25a","Type":"ContainerStarted","Data":"53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982"} Jul 20 15:54:06.658479 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.658295 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" event={"ID":"d5499e52-703f-4bb4-9ea8-1a3772c0d25a","Type":"ContainerStarted","Data":"882448d067e80df704d01d7b2bcea3174516f4b0590c3bbc8ac61bc842e9f031"} Jul 20 15:54:06.680526 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:06.680479 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podStartSLOduration=1.680460299 podStartE2EDuration="1.680460299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 15:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 15:54:06.678294043 +0000 UTC m=+1638.622887413" watchObservedRunningTime="2026-07-20 15:54:06.680460299 +0000 UTC m=+1638.625053674" Jul 20 15:54:07.151553 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:07.151510 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:07.153167 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:07.153141 2557 patch_prober.go:28] interesting pod/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 15:54:07.153290 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:07.153186 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 15:54:08.157131 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:08.157099 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:08.670856 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:08.670829 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:08.671986 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:08.671965 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:14.491976 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:14.491944 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:54:22.758522 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:22.758478 2557 generic.go:356] "Generic (PLEG): container finished" podID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerID="53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982" exitCode=0 Jul 20 15:54:22.758999 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:22.758551 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" event={"ID":"d5499e52-703f-4bb4-9ea8-1a3772c0d25a","Type":"ContainerDied","Data":"53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982"} Jul 20 15:54:22.758999 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:22.758962 2557 scope.go:117] "RemoveContainer" containerID="53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982" Jul 20 15:54:22.759986 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:22.759955 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:54:22.760095 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:22.760055 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:54:22.760157 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:22.760107 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 15:54:23.670760 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:23.670724 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:23.766616 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:23.766582 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" event={"ID":"d5499e52-703f-4bb4-9ea8-1a3772c0d25a","Type":"ContainerStarted","Data":"993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109"} Jul 20 15:54:24.152007 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:24.151971 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:24.153219 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:24.153191 2557 patch_prober.go:28] interesting pod/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 15:54:24.153337 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:24.153241 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 15:54:25.152005 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:25.151969 2557 patch_prober.go:28] interesting pod/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 15:54:25.152415 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:25.152044 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 15:54:26.152521 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:26.152478 2557 patch_prober.go:28] interesting pod/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 15:54:26.152940 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:26.152556 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 15:54:27.152273 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:27.152231 2557 patch_prober.go:28] interesting pod/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 15:54:27.152461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:27.152310 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 15:54:28.155461 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:28.155432 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:28.155857 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:28.155722 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:28.156403 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:28.156386 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:47.528578 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:47.528538 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:54:50.569806 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:50.569757 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6"] Jul 20 15:54:50.570217 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:50.569989 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" containerID="cri-o://993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109" gracePeriod=30 Jul 20 15:54:55.837145 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.837088 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:55.881113 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881080 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istiod-ca-cert\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881272 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881183 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-token\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881272 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881212 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtc6h\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-kube-api-access-jtc6h\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881272 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881247 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-credential-socket\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881444 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881279 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-podinfo\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881500 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881477 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-envoy\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881555 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881522 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-data\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881539 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:54:55.881610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881563 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 15:54:55.881610 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881571 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-certs\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881858 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881613 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-socket\") pod \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\" (UID: \"d5499e52-703f-4bb4-9ea8-1a3772c0d25a\") " Jul 20 15:54:55.881858 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881823 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:54:55.881976 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.881911 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:54:55.882074 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.882052 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.882143 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.882081 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istiod-ca-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.882143 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.882095 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-credential-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.882143 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.882109 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-workload-certs\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.882630 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.882600 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:54:55.883905 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.883871 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 15:54:55.883905 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.883894 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:54:55.884049 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.883950 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-kube-api-access-jtc6h" (OuterVolumeSpecName: "kube-api-access-jtc6h") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "kube-api-access-jtc6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 15:54:55.884049 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.884022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5499e52-703f-4bb4-9ea8-1a3772c0d25a" (UID: "d5499e52-703f-4bb4-9ea8-1a3772c0d25a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 15:54:55.955951 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.955924 2557 generic.go:356] "Generic (PLEG): container finished" podID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerID="993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109" exitCode=0 Jul 20 15:54:55.956128 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.955980 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" Jul 20 15:54:55.956128 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.956007 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" event={"ID":"d5499e52-703f-4bb4-9ea8-1a3772c0d25a","Type":"ContainerDied","Data":"993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109"} Jul 20 15:54:55.956128 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.956061 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6" event={"ID":"d5499e52-703f-4bb4-9ea8-1a3772c0d25a","Type":"ContainerDied","Data":"882448d067e80df704d01d7b2bcea3174516f4b0590c3bbc8ac61bc842e9f031"} Jul 20 15:54:55.956128 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.956085 2557 scope.go:117] "RemoveContainer" containerID="993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109" Jul 20 15:54:55.965864 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.965848 2557 scope.go:117] "RemoveContainer" containerID="53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982" Jul 20 15:54:55.973933 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.973910 2557 scope.go:117] "RemoveContainer" containerID="993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109" Jul 20 15:54:55.974163 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:54:55.974143 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109\": container with ID starting with 993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109 not found: ID does not exist" containerID="993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109" Jul 20 15:54:55.974210 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.974173 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109"} err="failed to get container status \"993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109\": rpc error: code = NotFound desc = could not find container \"993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109\": container with ID starting with 993e1fbebdfe318789509d3edb9d9bce67af93c14cfbbd59fbb51c8ddd00c109 not found: ID does not exist" Jul 20 15:54:55.974210 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.974192 2557 scope.go:117] "RemoveContainer" containerID="53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982" Jul 20 15:54:55.974422 ip-10-0-141-150 kubenswrapper[2557]: E0720 15:54:55.974406 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982\": container with ID starting with 53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982 not found: ID does not exist" containerID="53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982" Jul 20 15:54:55.974478 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.974430 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982"} err="failed to get container status \"53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982\": rpc error: code = NotFound desc = could not find container \"53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982\": container with ID starting with 53b35de118ab414a03fc3ae4f75c3ef6bc8df4f5946c7f7b38d2086de32cf982 not found: ID does not exist" Jul 20 15:54:55.978902 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.978882 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6"] Jul 20 15:54:55.982587 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.982568 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-podinfo\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.982666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.982589 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-envoy\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.982666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.982603 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-data\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.982666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.982613 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-istio-token\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.982666 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.982626 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtc6h\" (UniqueName: \"kubernetes.io/projected/d5499e52-703f-4bb4-9ea8-1a3772c0d25a-kube-api-access-jtc6h\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 15:54:55.986391 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:55.986368 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dde17afb-openshift-default-74c4d5db87-s26l6"] Jul 20 15:54:56.566447 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:56.566416 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" path="/var/lib/kubelet/pods/d5499e52-703f-4bb4-9ea8-1a3772c0d25a/volumes" Jul 20 15:54:59.590265 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:54:59.590227 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:55:32.788276 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:55:32.788197 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:56:48.824938 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:56:48.824909 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:56:48.830708 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:56:48.830687 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 15:57:00.983127 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:57:00.983090 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:57:10.290407 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:57:10.290371 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:57:25.313042 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:57:25.313003 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:57:46.590385 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:57:46.590351 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:58:28.686822 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:58:28.686785 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:58:40.578685 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:58:40.578648 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:58:50.475372 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:58:50.475337 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:59:08.586929 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:59:08.586897 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 15:59:16.970911 ip-10-0-141-150 kubenswrapper[2557]: I0720 15:59:16.970873 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:00:15.793057 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:00:15.793016 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:00:25.386850 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:00:25.386812 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:01:08.526563 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.526528 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724"] Jul 20 16:01:08.527189 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527115 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" Jul 20 16:01:08.527189 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527137 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" Jul 20 16:01:08.527189 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527167 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" Jul 20 16:01:08.527189 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527176 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a5673c1-9c29-4126-b495-c094bc4caf4a" containerName="istio-proxy" Jul 20 16:01:08.527189 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527184 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" Jul 20 16:01:08.527189 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527189 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" Jul 20 16:01:08.527458 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527270 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" Jul 20 16:01:08.527458 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.527282 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5499e52-703f-4bb4-9ea8-1a3772c0d25a" containerName="istio-proxy" Jul 20 16:01:08.530390 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.530372 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.532820 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.532798 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-48c284-openshift-default-dockercfg-mncmt\"" Jul 20 16:01:08.548258 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.548232 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724"] Jul 20 16:01:08.624343 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-socket\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624539 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624356 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-credential-socket\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624539 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istiod-ca-cert\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624539 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624526 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzdrw\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-kube-api-access-kzdrw\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624722 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624581 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-podinfo\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624722 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624608 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-token\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624722 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624658 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-data\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624722 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624689 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-certs\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.624950 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.624841 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-envoy\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726249 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istiod-ca-cert\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726406 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzdrw\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-kube-api-access-kzdrw\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726406 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-podinfo\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726406 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726337 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-token\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726406 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726367 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-data\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726622 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-certs\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726622 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-envoy\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726622 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-socket\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726622 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-credential-socket\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726870 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-data\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726934 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726875 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-certs\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.726986 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.726932 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-credential-socket\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.727064 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.727038 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istiod-ca-cert\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.727123 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.727056 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-socket\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.728889 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.728864 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-envoy\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.729066 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.729048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-podinfo\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.736123 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.736094 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-token\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.736322 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.736306 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzdrw\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-kube-api-access-kzdrw\") pod \"e2e-inf-a-48c284-openshift-default-675b4446c5-rf724\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.840667 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.840592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:08.998426 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:08.998383 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724"] Jul 20 16:01:08.998536 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:01:08.998479 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d26705_6e61_4da5_81fe_6fc5442d38a6.slice/crio-0bfcc5f6ceb7a8ceadbc7326d2a30c5392335f4d10f59bc499ab5fb55e1f685f WatchSource:0}: Error finding container 0bfcc5f6ceb7a8ceadbc7326d2a30c5392335f4d10f59bc499ab5fb55e1f685f: Status 404 returned error can't find the container with id 0bfcc5f6ceb7a8ceadbc7326d2a30c5392335f4d10f59bc499ab5fb55e1f685f Jul 20 16:01:09.000803 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:09.000743 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:01:09.000908 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:09.000879 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:01:09.000965 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:09.000948 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:01:09.041022 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:09.040991 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" event={"ID":"d4d26705-6e61-4da5-81fe-6fc5442d38a6","Type":"ContainerStarted","Data":"0bfcc5f6ceb7a8ceadbc7326d2a30c5392335f4d10f59bc499ab5fb55e1f685f"} Jul 20 16:01:10.048164 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:10.048122 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" event={"ID":"d4d26705-6e61-4da5-81fe-6fc5442d38a6","Type":"ContainerStarted","Data":"afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de"} Jul 20 16:01:10.069609 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:10.069560 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" podStartSLOduration=2.06954771 podStartE2EDuration="2.06954771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:01:10.067367275 +0000 UTC m=+2062.011960647" watchObservedRunningTime="2026-07-20 16:01:10.06954771 +0000 UTC m=+2062.014141080" Jul 20 16:01:10.841489 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:10.841453 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:10.846622 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:10.846597 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:11.053748 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:11.053718 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:11.054682 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:11.054664 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:16.477731 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:16.477701 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:01:24.132279 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:24.132252 2557 generic.go:356] "Generic (PLEG): container finished" podID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerID="afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de" exitCode=0 Jul 20 16:01:24.132626 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:24.132320 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" event={"ID":"d4d26705-6e61-4da5-81fe-6fc5442d38a6","Type":"ContainerDied","Data":"afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de"} Jul 20 16:01:24.132719 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:24.132704 2557 scope.go:117] "RemoveContainer" containerID="afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de" Jul 20 16:01:24.133741 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:24.133708 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:01:24.133836 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:24.133819 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:01:24.133874 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:24.133863 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:01:25.143675 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:25.143631 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" event={"ID":"d4d26705-6e61-4da5-81fe-6fc5442d38a6","Type":"ContainerStarted","Data":"61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834"} Jul 20 16:01:25.841025 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:25.840990 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:25.842463 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:25.842432 2557 patch_prober.go:28] interesting pod/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 16:01:25.842603 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:25.842495 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 16:01:26.054012 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:26.053980 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:26.841067 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:26.841019 2557 patch_prober.go:28] interesting pod/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 16:01:26.841578 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:26.841092 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 16:01:27.841198 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:27.841154 2557 patch_prober.go:28] interesting pod/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 16:01:27.841687 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:27.841246 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 16:01:28.842090 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:28.841657 2557 patch_prober.go:28] interesting pod/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 16:01:28.842090 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:28.841716 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 16:01:29.844630 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:29.844598 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:29.845597 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:29.845579 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:01:29.893454 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:29.893417 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:01:43.688835 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:43.688797 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:01:48.860074 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:48.860042 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 16:01:48.867667 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:01:48.867647 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 16:02:08.889738 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:02:08.889705 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:02:22.697570 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:02:22.697534 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:02:49.776709 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:02:49.776628 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:03:01.689576 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:01.689538 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:03:08.115500 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:08.115464 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724"] Jul 20 16:03:08.116048 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:08.115665 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" containerID="cri-o://61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834" gracePeriod=30 Jul 20 16:03:13.366329 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.366303 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:03:13.452101 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452035 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-data\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452101 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452094 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzdrw\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-kube-api-access-kzdrw\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452287 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452119 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-certs\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452287 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452164 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istiod-ca-cert\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452287 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452202 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-envoy\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452287 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452242 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-token\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452287 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452269 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-credential-socket\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452539 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452301 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-socket\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452539 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452341 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-podinfo\") pod \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\" (UID: \"d4d26705-6e61-4da5-81fe-6fc5442d38a6\") " Jul 20 16:03:13.452539 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452437 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:03:13.452681 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452592 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:03:13.452681 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452656 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:03:13.452815 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452728 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-credential-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.452815 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452748 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-certs\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.452915 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452831 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:03:13.452954 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.452926 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:03:13.454510 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.454479 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:03:13.454628 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.454514 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-kube-api-access-kzdrw" (OuterVolumeSpecName: "kube-api-access-kzdrw") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "kube-api-access-kzdrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:03:13.454835 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.454818 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:03:13.454966 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.454949 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4d26705-6e61-4da5-81fe-6fc5442d38a6" (UID: "d4d26705-6e61-4da5-81fe-6fc5442d38a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:03:13.553522 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553494 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-data\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.553522 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553521 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzdrw\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-kube-api-access-kzdrw\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.553679 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553530 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istiod-ca-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.553679 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553539 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-envoy\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.553679 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553547 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-token\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.553679 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553555 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4d26705-6e61-4da5-81fe-6fc5442d38a6-workload-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.553679 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.553562 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4d26705-6e61-4da5-81fe-6fc5442d38a6-istio-podinfo\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:03:13.764027 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.763940 2557 generic.go:356] "Generic (PLEG): container finished" podID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerID="61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834" exitCode=0 Jul 20 16:03:13.764027 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.763989 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" Jul 20 16:03:13.764027 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.763998 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" event={"ID":"d4d26705-6e61-4da5-81fe-6fc5442d38a6","Type":"ContainerDied","Data":"61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834"} Jul 20 16:03:13.764302 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.764040 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724" event={"ID":"d4d26705-6e61-4da5-81fe-6fc5442d38a6","Type":"ContainerDied","Data":"0bfcc5f6ceb7a8ceadbc7326d2a30c5392335f4d10f59bc499ab5fb55e1f685f"} Jul 20 16:03:13.764302 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.764061 2557 scope.go:117] "RemoveContainer" containerID="61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834" Jul 20 16:03:13.774048 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.774029 2557 scope.go:117] "RemoveContainer" containerID="afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de" Jul 20 16:03:13.782252 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.782232 2557 scope.go:117] "RemoveContainer" containerID="61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834" Jul 20 16:03:13.782511 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:03:13.782492 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834\": container with ID starting with 61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834 not found: ID does not exist" containerID="61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834" Jul 20 16:03:13.782561 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.782519 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834"} err="failed to get container status \"61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834\": rpc error: code = NotFound desc = could not find container \"61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834\": container with ID starting with 61e753e57f2456cbbe140043f0295c20bc3708310366d54401171ee0d27a7834 not found: ID does not exist" Jul 20 16:03:13.782561 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.782536 2557 scope.go:117] "RemoveContainer" containerID="afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de" Jul 20 16:03:13.782830 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:03:13.782809 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de\": container with ID starting with afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de not found: ID does not exist" containerID="afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de" Jul 20 16:03:13.782924 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.782835 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de"} err="failed to get container status \"afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de\": rpc error: code = NotFound desc = could not find container \"afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de\": container with ID starting with afcd3a607468a99fa889feb34dc0e45bed37c593b9d886a5cb375ad1b70e92de not found: ID does not exist" Jul 20 16:03:13.786168 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.786143 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724"] Jul 20 16:03:13.791281 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:13.791260 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-48c284-openshift-default-675b4446c5-rf724"] Jul 20 16:03:14.565700 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:14.565669 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" path="/var/lib/kubelet/pods/d4d26705-6e61-4da5-81fe-6fc5442d38a6/volumes" Jul 20 16:03:30.890440 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:30.890404 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:03:44.348234 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.348200 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk"] Jul 20 16:03:44.348689 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.348584 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" Jul 20 16:03:44.348689 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.348596 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" Jul 20 16:03:44.348689 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.348605 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" Jul 20 16:03:44.348689 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.348611 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" Jul 20 16:03:44.348689 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.348666 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" Jul 20 16:03:44.351901 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.351876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.354966 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.354947 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5a8c68-openshift-default-dockercfg-jwmrt\"" Jul 20 16:03:44.371048 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.371013 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk"] Jul 20 16:03:44.428167 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428143 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9de7ed-9028-454e-b366-e42f8f0861b4-istiod-ca-cert\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428324 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428181 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-podinfo\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428324 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-token\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428449 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428345 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nscbt\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-kube-api-access-nscbt\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428449 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428398 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-socket\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428449 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428443 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-credential-socket\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428610 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-envoy\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428610 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428506 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-data\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.428610 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.428540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-certs\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.529739 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.529704 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-envoy\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.529739 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.529741 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-data\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.529996 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.529877 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-certs\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.529996 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.529930 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9de7ed-9028-454e-b366-e42f8f0861b4-istiod-ca-cert\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.529996 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.529959 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-podinfo\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.529996 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.529976 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-token\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530208 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nscbt\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-kube-api-access-nscbt\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530208 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530114 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-socket\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530208 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530160 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-credential-socket\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530361 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-certs\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530424 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530115 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-data\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530482 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530450 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-credential-socket\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530686 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9de7ed-9028-454e-b366-e42f8f0861b4-istiod-ca-cert\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.530824 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.530723 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-socket\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.532094 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.532077 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-envoy\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.532946 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.532924 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-podinfo\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.541298 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.541274 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nscbt\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-kube-api-access-nscbt\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.542272 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.542250 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-token\") pod \"e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.662502 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.662474 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:44.809006 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.808980 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk"] Jul 20 16:03:44.810044 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:03:44.810007 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9de7ed_9028_454e_b366_e42f8f0861b4.slice/crio-59798b1b11fe445223451dbb3f897b187528cb144e65c7cbbca9304b643e775d WatchSource:0}: Error finding container 59798b1b11fe445223451dbb3f897b187528cb144e65c7cbbca9304b643e775d: Status 404 returned error can't find the container with id 59798b1b11fe445223451dbb3f897b187528cb144e65c7cbbca9304b643e775d Jul 20 16:03:44.812159 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.812127 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:03:44.812237 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.812191 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:03:44.812237 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.812218 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:03:44.950807 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.950703 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" event={"ID":"3b9de7ed-9028-454e-b366-e42f8f0861b4","Type":"ContainerStarted","Data":"74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4"} Jul 20 16:03:44.950807 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.950766 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" event={"ID":"3b9de7ed-9028-454e-b366-e42f8f0861b4","Type":"ContainerStarted","Data":"59798b1b11fe445223451dbb3f897b187528cb144e65c7cbbca9304b643e775d"} Jul 20 16:03:44.982400 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:44.977816 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podStartSLOduration=0.977789357 podStartE2EDuration="977.789357ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:03:44.971538253 +0000 UTC m=+2216.916131622" watchObservedRunningTime="2026-07-20 16:03:44.977789357 +0000 UTC m=+2216.922382724" Jul 20 16:03:45.662984 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:45.662950 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:45.667700 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:45.667676 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:45.957483 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:45.957390 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:45.958274 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:45.958253 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:03:53.202303 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:03:53.202271 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:04:00.966822 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:00.965837 2557 patch_prober.go:28] interesting pod/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:39834->10.133.0.58:15021: read: connection reset by peer" start-of-body= Jul 20 16:04:00.966822 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:00.965920 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:39834->10.133.0.58:15021: read: connection reset by peer" Jul 20 16:04:01.057179 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:01.057144 2557 generic.go:356] "Generic (PLEG): container finished" podID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerID="74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4" exitCode=0 Jul 20 16:04:01.057367 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:01.057185 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" event={"ID":"3b9de7ed-9028-454e-b366-e42f8f0861b4","Type":"ContainerDied","Data":"74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4"} Jul 20 16:04:01.057648 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:01.057629 2557 scope.go:117] "RemoveContainer" containerID="74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4" Jul 20 16:04:01.058687 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:01.058659 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:04:01.058757 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:01.058727 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:04:01.058818 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:01.058758 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:04:02.064641 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:02.064608 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" event={"ID":"3b9de7ed-9028-454e-b366-e42f8f0861b4","Type":"ContainerStarted","Data":"b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f"} Jul 20 16:04:02.663663 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:02.663625 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:04:02.663931 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:02.663906 2557 patch_prober.go:28] interesting pod/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 16:04:02.663982 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:02.663956 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 16:04:03.663404 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:03.663363 2557 patch_prober.go:28] interesting pod/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 16:04:03.663850 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:03.663422 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 16:04:04.662900 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:04.662858 2557 patch_prober.go:28] interesting pod/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 16:04:04.663096 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:04.662915 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 16:04:05.662939 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:05.662899 2557 patch_prober.go:28] interesting pod/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 16:04:05.663314 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:05.662994 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 16:04:06.666887 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:06.666851 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:04:06.667354 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:06.667172 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:04:06.667757 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:06.667737 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:04:08.990641 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:08.990599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:04:27.878467 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:27.878432 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:04:35.834177 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:35.834135 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk"] Jul 20 16:04:35.834639 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:35.834385 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" containerID="cri-o://b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f" gracePeriod=30 Jul 20 16:04:36.669727 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:36.669694 2557 patch_prober.go:28] interesting pod/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 16:04:36.669912 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:36.669749 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 16:04:41.077754 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.077730 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:04:41.266391 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266360 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-envoy\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266563 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266410 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-token\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266563 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266468 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9de7ed-9028-454e-b366-e42f8f0861b4-istiod-ca-cert\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266563 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266485 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nscbt\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-kube-api-access-nscbt\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266563 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266525 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-credential-socket\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266817 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266568 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-socket\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266817 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266596 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-certs\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266817 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266621 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-podinfo\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.266817 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266649 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-data\") pod \"3b9de7ed-9028-454e-b366-e42f8f0861b4\" (UID: \"3b9de7ed-9028-454e-b366-e42f8f0861b4\") " Jul 20 16:04:41.267035 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266904 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:04:41.267035 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266915 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:04:41.267035 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.266922 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9de7ed-9028-454e-b366-e42f8f0861b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:04:41.267200 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.267049 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:04:41.267913 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.267888 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:04:41.268798 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.268752 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:04:41.269035 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.269005 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-kube-api-access-nscbt" (OuterVolumeSpecName: "kube-api-access-nscbt") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "kube-api-access-nscbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:04:41.269209 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.269195 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:04:41.269348 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.269326 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b9de7ed-9028-454e-b366-e42f8f0861b4" (UID: "3b9de7ed-9028-454e-b366-e42f8f0861b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:04:41.286101 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.286077 2557 generic.go:356] "Generic (PLEG): container finished" podID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerID="b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f" exitCode=0 Jul 20 16:04:41.286216 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.286123 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" Jul 20 16:04:41.286216 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.286153 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" event={"ID":"3b9de7ed-9028-454e-b366-e42f8f0861b4","Type":"ContainerDied","Data":"b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f"} Jul 20 16:04:41.286216 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.286208 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk" event={"ID":"3b9de7ed-9028-454e-b366-e42f8f0861b4","Type":"ContainerDied","Data":"59798b1b11fe445223451dbb3f897b187528cb144e65c7cbbca9304b643e775d"} Jul 20 16:04:41.286352 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.286230 2557 scope.go:117] "RemoveContainer" containerID="b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f" Jul 20 16:04:41.296071 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.296050 2557 scope.go:117] "RemoveContainer" containerID="74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4" Jul 20 16:04:41.304221 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.304206 2557 scope.go:117] "RemoveContainer" containerID="b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f" Jul 20 16:04:41.304438 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:04:41.304423 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f\": container with ID starting with b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f not found: ID does not exist" containerID="b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f" Jul 20 16:04:41.304483 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.304444 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f"} err="failed to get container status \"b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f\": rpc error: code = NotFound desc = could not find container \"b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f\": container with ID starting with b2dc5dd8d17ca76b30f6d583e0804984457270bbe50dbf9935bb88a17d949b7f not found: ID does not exist" Jul 20 16:04:41.304483 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.304458 2557 scope.go:117] "RemoveContainer" containerID="74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4" Jul 20 16:04:41.304704 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:04:41.304686 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4\": container with ID starting with 74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4 not found: ID does not exist" containerID="74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4" Jul 20 16:04:41.304762 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.304712 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4"} err="failed to get container status \"74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4\": rpc error: code = NotFound desc = could not find container \"74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4\": container with ID starting with 74fd0e61f89440d27461db55f81aaeb1550b55103b7af8bd7d706159294af2a4 not found: ID does not exist" Jul 20 16:04:41.308109 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.308084 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk"] Jul 20 16:04:41.311661 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.311640 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5a8c68-openshift-default-6d7d597ddf-wtqjk"] Jul 20 16:04:41.367552 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367523 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9de7ed-9028-454e-b366-e42f8f0861b4-istiod-ca-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367552 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367548 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nscbt\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-kube-api-access-nscbt\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367565 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-credential-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367578 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367591 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-workload-certs\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367603 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-podinfo\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367615 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-data\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367628 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-envoy\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:41.367701 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:41.367640 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9de7ed-9028-454e-b366-e42f8f0861b4-istio-token\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:04:42.566163 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:42.566132 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" path="/var/lib/kubelet/pods/3b9de7ed-9028-454e-b366-e42f8f0861b4/volumes" Jul 20 16:04:56.599856 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:04:56.599821 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:05:04.819421 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.819389 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr"] Jul 20 16:05:04.819889 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.819834 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" Jul 20 16:05:04.819889 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.819848 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" Jul 20 16:05:04.819994 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.819923 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" Jul 20 16:05:04.819994 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.819931 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4d26705-6e61-4da5-81fe-6fc5442d38a6" containerName="istio-proxy" Jul 20 16:05:04.819994 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.819941 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" Jul 20 16:05:04.822298 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.822279 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.825700 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.825674 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8791e0-openshift-default-dockercfg-t9v6k\"" Jul 20 16:05:04.841672 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.841646 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr"] Jul 20 16:05:04.859047 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859020 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-credential-socket\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859155 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e03a4a-861e-435a-b2ac-0339b1f352fa-istiod-ca-cert\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859155 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859093 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-certs\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859244 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859163 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-envoy\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859244 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859231 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhd6\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-kube-api-access-dmhd6\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859325 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-podinfo\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859395 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859371 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-socket\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859456 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859433 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-data\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.859508 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.859474 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-token\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960216 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960188 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-token\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960412 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960237 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-credential-socket\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960412 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e03a4a-861e-435a-b2ac-0339b1f352fa-istiod-ca-cert\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960412 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-certs\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960412 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960332 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-envoy\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960412 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960372 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhd6\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-kube-api-access-dmhd6\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960412 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960411 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-podinfo\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960748 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960487 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-socket\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960748 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960535 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-data\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960748 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960647 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-credential-socket\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960748 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960727 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-certs\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.960958 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960901 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-socket\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.961013 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.960993 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-data\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.961109 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.961087 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e03a4a-861e-435a-b2ac-0339b1f352fa-istiod-ca-cert\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.962914 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.962895 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-envoy\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.963139 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.963119 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-podinfo\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.971712 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.971686 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhd6\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-kube-api-access-dmhd6\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:04.972222 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:04.972205 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-token\") pod \"e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:05.132852 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.132820 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:05.295180 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.295153 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr"] Jul 20 16:05:05.296587 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:05:05.296562 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e03a4a_861e_435a_b2ac_0339b1f352fa.slice/crio-4a348fa161646e947e2f6141f9b1ee9a4686139cc26daf0b996f0480511c30c0 WatchSource:0}: Error finding container 4a348fa161646e947e2f6141f9b1ee9a4686139cc26daf0b996f0480511c30c0: Status 404 returned error can't find the container with id 4a348fa161646e947e2f6141f9b1ee9a4686139cc26daf0b996f0480511c30c0 Jul 20 16:05:05.298746 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.298712 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:05:05.298863 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.298804 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:05:05.298863 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.298841 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:05:05.430175 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.430092 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" event={"ID":"55e03a4a-861e-435a-b2ac-0339b1f352fa","Type":"ContainerStarted","Data":"a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127"} Jul 20 16:05:05.430175 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.430161 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" event={"ID":"55e03a4a-861e-435a-b2ac-0339b1f352fa","Type":"ContainerStarted","Data":"4a348fa161646e947e2f6141f9b1ee9a4686139cc26daf0b996f0480511c30c0"} Jul 20 16:05:05.460659 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:05.460598 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podStartSLOduration=1.460576378 podStartE2EDuration="1.460576378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:05:05.455292365 +0000 UTC m=+2297.399885732" watchObservedRunningTime="2026-07-20 16:05:05.460576378 +0000 UTC m=+2297.405169750" Jul 20 16:05:06.133138 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:06.133096 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:06.137564 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:06.137538 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:06.437647 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:06.437566 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:06.438844 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:06.438818 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:15.093914 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:15.093872 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:05:22.478961 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:22.478918 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 16:05:22.479365 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:22.479004 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 16:05:23.538021 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:23.537984 2557 generic.go:356] "Generic (PLEG): container finished" podID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerID="a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127" exitCode=0 Jul 20 16:05:23.538552 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:23.538061 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" event={"ID":"55e03a4a-861e-435a-b2ac-0339b1f352fa","Type":"ContainerDied","Data":"a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127"} Jul 20 16:05:23.538619 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:23.538605 2557 scope.go:117] "RemoveContainer" containerID="a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127" Jul 20 16:05:23.539671 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:23.539644 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:05:23.539808 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:23.539729 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:05:23.539808 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:23.539760 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 20 16:05:24.549083 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:24.549040 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" event={"ID":"55e03a4a-861e-435a-b2ac-0339b1f352fa","Type":"ContainerStarted","Data":"cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11"} Jul 20 16:05:25.133277 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:25.133232 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:25.133583 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:25.133554 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 16:05:25.133680 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:25.133621 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 16:05:26.133871 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:26.133829 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 16:05:26.134342 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:26.133913 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 16:05:27.134215 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:27.134176 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 16:05:27.134713 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:27.134264 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 16:05:28.134146 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:28.134107 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 16:05:28.134447 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:28.134416 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 16:05:28.782897 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:28.782865 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:05:29.136855 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:29.136825 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:29.137354 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:29.137338 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:29.137993 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:29.137974 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:05:40.172212 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:05:40.172131 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:06:13.473277 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:06:13.473233 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:06:26.878167 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:06:26.878126 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:06:45.092582 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:06:45.092546 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:06:48.897373 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:06:48.897345 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 16:06:48.912426 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:06:48.912400 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 16:07:01.676010 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:07:01.675970 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:07:19.980495 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:07:19.980405 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:07:35.476044 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:07:35.476007 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:07:50.376461 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:07:50.376425 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:08:07.684210 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:08:07.684169 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:08:39.182197 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:08:39.182110 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:09:04.085569 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:09:04.085533 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:09:17.905041 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:09:17.905003 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:09:35.455814 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:09:35.455758 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:09:52.385146 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:09:52.385108 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:10:10.884855 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:10.881969 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:10:24.592538 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:24.592501 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:10:39.492228 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:39.492193 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:10:48.782463 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.782430 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7638-139999460064816"] Jul 20 16:10:48.783005 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.782984 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" Jul 20 16:10:48.783089 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.783006 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9de7ed-9028-454e-b366-e42f8f0861b4" containerName="istio-proxy" Jul 20 16:10:48.785202 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.785181 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:48.787109 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.787087 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wxz6v\"" Jul 20 16:10:48.790503 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.790481 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999460064816"] Jul 20 16:10:48.853801 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.853735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8czj5\" (UniqueName: \"kubernetes.io/projected/d7092f2d-acb1-429c-88da-446ff9c8786c-kube-api-access-8czj5\") pod \"test-curl-7638-139999460064816\" (UID: \"d7092f2d-acb1-429c-88da-446ff9c8786c\") " pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:48.954814 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.954790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8czj5\" (UniqueName: \"kubernetes.io/projected/d7092f2d-acb1-429c-88da-446ff9c8786c-kube-api-access-8czj5\") pod \"test-curl-7638-139999460064816\" (UID: \"d7092f2d-acb1-429c-88da-446ff9c8786c\") " pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:48.963431 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:48.963404 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8czj5\" (UniqueName: \"kubernetes.io/projected/d7092f2d-acb1-429c-88da-446ff9c8786c-kube-api-access-8czj5\") pod \"test-curl-7638-139999460064816\" (UID: \"d7092f2d-acb1-429c-88da-446ff9c8786c\") " pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:49.096658 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:49.096587 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:49.232598 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:49.232567 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999460064816"] Jul 20 16:10:49.235526 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:10:49.235503 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7092f2d_acb1_429c_88da_446ff9c8786c.slice/crio-c7e3fa3f215c2d8662f17b5e6f8d5a6931b656814e025f47fabc3b029b419f9a WatchSource:0}: Error finding container c7e3fa3f215c2d8662f17b5e6f8d5a6931b656814e025f47fabc3b029b419f9a: Status 404 returned error can't find the container with id c7e3fa3f215c2d8662f17b5e6f8d5a6931b656814e025f47fabc3b029b419f9a Jul 20 16:10:49.237285 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:49.237266 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:10:49.392360 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:49.392329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999460064816" event={"ID":"d7092f2d-acb1-429c-88da-446ff9c8786c","Type":"ContainerStarted","Data":"c7e3fa3f215c2d8662f17b5e6f8d5a6931b656814e025f47fabc3b029b419f9a"} Jul 20 16:10:50.400264 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:50.400229 2557 generic.go:356] "Generic (PLEG): container finished" podID="d7092f2d-acb1-429c-88da-446ff9c8786c" containerID="e041a641850a69ecae437fa64e521e0bc11dc04e2d7246d29361a7b0af8ff02e" exitCode=0 Jul 20 16:10:50.400264 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:50.400267 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999460064816" event={"ID":"d7092f2d-acb1-429c-88da-446ff9c8786c","Type":"ContainerDied","Data":"e041a641850a69ecae437fa64e521e0bc11dc04e2d7246d29361a7b0af8ff02e"} Jul 20 16:10:51.543400 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.543379 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:51.579747 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.579717 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8czj5\" (UniqueName: \"kubernetes.io/projected/d7092f2d-acb1-429c-88da-446ff9c8786c-kube-api-access-8czj5\") pod \"d7092f2d-acb1-429c-88da-446ff9c8786c\" (UID: \"d7092f2d-acb1-429c-88da-446ff9c8786c\") " Jul 20 16:10:51.582103 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.582070 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7092f2d-acb1-429c-88da-446ff9c8786c-kube-api-access-8czj5" (OuterVolumeSpecName: "kube-api-access-8czj5") pod "d7092f2d-acb1-429c-88da-446ff9c8786c" (UID: "d7092f2d-acb1-429c-88da-446ff9c8786c"). InnerVolumeSpecName "kube-api-access-8czj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:10:51.587091 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.587065 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7638-139999460064816_d7092f2d-acb1-429c-88da-446ff9c8786c/test-curl-7638-139999460064816/0.log" Jul 20 16:10:51.635847 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.635799 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7638-139999460064816"] Jul 20 16:10:51.644004 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.643971 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7638-139999460064816"] Jul 20 16:10:51.681241 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.681157 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8czj5\" (UniqueName: \"kubernetes.io/projected/d7092f2d-acb1-429c-88da-446ff9c8786c-kube-api-access-8czj5\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:10:51.768764 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.768723 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7638-139999460064432"] Jul 20 16:10:51.769175 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.769160 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7092f2d-acb1-429c-88da-446ff9c8786c" containerName="test-curl-7638-139999460064816" Jul 20 16:10:51.769243 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.769178 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7092f2d-acb1-429c-88da-446ff9c8786c" containerName="test-curl-7638-139999460064816" Jul 20 16:10:51.769286 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.769259 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7092f2d-acb1-429c-88da-446ff9c8786c" containerName="test-curl-7638-139999460064816" Jul 20 16:10:51.771234 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.771217 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:51.778890 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.778669 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999460064432"] Jul 20 16:10:51.883041 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.883007 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcdm2\" (UniqueName: \"kubernetes.io/projected/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846-kube-api-access-zcdm2\") pod \"test-curl-7638-139999460064432\" (UID: \"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846\") " pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:51.983964 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.983866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcdm2\" (UniqueName: \"kubernetes.io/projected/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846-kube-api-access-zcdm2\") pod \"test-curl-7638-139999460064432\" (UID: \"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846\") " pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:51.991694 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:51.991661 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcdm2\" (UniqueName: \"kubernetes.io/projected/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846-kube-api-access-zcdm2\") pod \"test-curl-7638-139999460064432\" (UID: \"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846\") " pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:52.082091 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:52.082061 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:52.214870 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:52.214826 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999460064432"] Jul 20 16:10:52.217437 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:10:52.217404 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d83c3cb_fe5b_4ebe_86e7_b7e07d081846.slice/crio-5c19991bc76c6edba9c2d3373e9db7714fa64f48a4bb682a2dd1420526328e03 WatchSource:0}: Error finding container 5c19991bc76c6edba9c2d3373e9db7714fa64f48a4bb682a2dd1420526328e03: Status 404 returned error can't find the container with id 5c19991bc76c6edba9c2d3373e9db7714fa64f48a4bb682a2dd1420526328e03 Jul 20 16:10:52.413178 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:52.413149 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:52.413356 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:52.413192 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7e3fa3f215c2d8662f17b5e6f8d5a6931b656814e025f47fabc3b029b419f9a" Jul 20 16:10:52.414423 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:52.414394 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999460064432" event={"ID":"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846","Type":"ContainerStarted","Data":"5c19991bc76c6edba9c2d3373e9db7714fa64f48a4bb682a2dd1420526328e03"} Jul 20 16:10:52.423278 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:10:52.423254 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999460064816\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="d7092f2d-acb1-429c-88da-446ff9c8786c" pod="opendatahub/test-curl-7638-139999460064816" Jul 20 16:10:52.566320 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:52.566292 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7092f2d-acb1-429c-88da-446ff9c8786c" path="/var/lib/kubelet/pods/d7092f2d-acb1-429c-88da-446ff9c8786c/volumes" Jul 20 16:10:53.422259 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:53.422222 2557 generic.go:356] "Generic (PLEG): container finished" podID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" containerID="6db7b4a2dc0b13ba77f2ffc497937ac555146e19eec281617ffdb16822f86d77" exitCode=0 Jul 20 16:10:53.422404 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:53.422281 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999460064432" event={"ID":"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846","Type":"ContainerDied","Data":"6db7b4a2dc0b13ba77f2ffc497937ac555146e19eec281617ffdb16822f86d77"} Jul 20 16:10:53.673444 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:53.673353 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:10:54.564846 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.564824 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:54.585968 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.585930 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7638-139999460064432_0d83c3cb-fe5b-4ebe-86e7-b7e07d081846/test-curl-7638-139999460064432/0.log" Jul 20 16:10:54.609397 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.609361 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcdm2\" (UniqueName: \"kubernetes.io/projected/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846-kube-api-access-zcdm2\") pod \"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846\" (UID: \"0d83c3cb-fe5b-4ebe-86e7-b7e07d081846\") " Jul 20 16:10:54.611490 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.611464 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846-kube-api-access-zcdm2" (OuterVolumeSpecName: "kube-api-access-zcdm2") pod "0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" (UID: "0d83c3cb-fe5b-4ebe-86e7-b7e07d081846"). InnerVolumeSpecName "kube-api-access-zcdm2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:10:54.615865 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.615843 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7638-139999460064432"] Jul 20 16:10:54.620106 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.620084 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7638-139999460064432"] Jul 20 16:10:54.710952 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.710870 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcdm2\" (UniqueName: \"kubernetes.io/projected/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846-kube-api-access-zcdm2\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:10:54.795185 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.795152 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:54.795634 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.795613 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" containerName="test-curl-7638-139999460064432" Jul 20 16:10:54.795725 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.795637 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" containerName="test-curl-7638-139999460064432" Jul 20 16:10:54.795803 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.795765 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" containerName="test-curl-7638-139999460064432" Jul 20 16:10:54.798785 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.798749 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:54.815889 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.815863 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:54.913843 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:54.913809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g9bj\" (UniqueName: \"kubernetes.io/projected/135c218b-48cc-4e67-8d62-2de1e7999c7e-kube-api-access-7g9bj\") pod \"test-curl-7638-139999459902640\" (UID: \"135c218b-48cc-4e67-8d62-2de1e7999c7e\") " pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:55.014737 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.014641 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g9bj\" (UniqueName: \"kubernetes.io/projected/135c218b-48cc-4e67-8d62-2de1e7999c7e-kube-api-access-7g9bj\") pod \"test-curl-7638-139999459902640\" (UID: \"135c218b-48cc-4e67-8d62-2de1e7999c7e\") " pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:55.026478 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.026441 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g9bj\" (UniqueName: \"kubernetes.io/projected/135c218b-48cc-4e67-8d62-2de1e7999c7e-kube-api-access-7g9bj\") pod \"test-curl-7638-139999459902640\" (UID: \"135c218b-48cc-4e67-8d62-2de1e7999c7e\") " pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:55.107755 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.107722 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:55.245203 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.245178 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:55.247657 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:10:55.247631 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod135c218b_48cc_4e67_8d62_2de1e7999c7e.slice/crio-424cf443d36e45a80ccbb7b557ad43c6b7b8ef480398f256fbfaf74bcba75b16 WatchSource:0}: Error finding container 424cf443d36e45a80ccbb7b557ad43c6b7b8ef480398f256fbfaf74bcba75b16: Status 404 returned error can't find the container with id 424cf443d36e45a80ccbb7b557ad43c6b7b8ef480398f256fbfaf74bcba75b16 Jul 20 16:10:55.435131 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.435096 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:55.435131 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.435113 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c19991bc76c6edba9c2d3373e9db7714fa64f48a4bb682a2dd1420526328e03" Jul 20 16:10:55.436288 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:55.436248 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459902640" event={"ID":"135c218b-48cc-4e67-8d62-2de1e7999c7e","Type":"ContainerStarted","Data":"424cf443d36e45a80ccbb7b557ad43c6b7b8ef480398f256fbfaf74bcba75b16"} Jul 20 16:10:55.445313 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:10:55.445285 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999460064432\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:56.444474 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:56.444446 2557 generic.go:356] "Generic (PLEG): container finished" podID="135c218b-48cc-4e67-8d62-2de1e7999c7e" containerID="2cba2c19a7d9d7e18ecbb59dec7b130385d5d7e893e04fbf1bf504b6f22bdb5d" exitCode=0 Jul 20 16:10:56.444931 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:56.444534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459902640" event={"ID":"135c218b-48cc-4e67-8d62-2de1e7999c7e","Type":"ContainerDied","Data":"2cba2c19a7d9d7e18ecbb59dec7b130385d5d7e893e04fbf1bf504b6f22bdb5d"} Jul 20 16:10:56.477701 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:10:56.477664 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999460064432\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" pod="opendatahub/test-curl-7638-139999460064432" Jul 20 16:10:56.565914 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:56.565882 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d83c3cb-fe5b-4ebe-86e7-b7e07d081846" path="/var/lib/kubelet/pods/0d83c3cb-fe5b-4ebe-86e7-b7e07d081846/volumes" Jul 20 16:10:57.585612 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.585589 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:57.607907 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.607880 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7638-139999459902640_135c218b-48cc-4e67-8d62-2de1e7999c7e/test-curl-7638-139999459902640/0.log" Jul 20 16:10:57.640210 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.640181 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g9bj\" (UniqueName: \"kubernetes.io/projected/135c218b-48cc-4e67-8d62-2de1e7999c7e-kube-api-access-7g9bj\") pod \"135c218b-48cc-4e67-8d62-2de1e7999c7e\" (UID: \"135c218b-48cc-4e67-8d62-2de1e7999c7e\") " Jul 20 16:10:57.642469 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.642440 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/135c218b-48cc-4e67-8d62-2de1e7999c7e-kube-api-access-7g9bj" (OuterVolumeSpecName: "kube-api-access-7g9bj") pod "135c218b-48cc-4e67-8d62-2de1e7999c7e" (UID: "135c218b-48cc-4e67-8d62-2de1e7999c7e"). InnerVolumeSpecName "kube-api-access-7g9bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:10:57.651985 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.651961 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:57.656680 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.656660 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:57.742259 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.742175 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g9bj\" (UniqueName: \"kubernetes.io/projected/135c218b-48cc-4e67-8d62-2de1e7999c7e-kube-api-access-7g9bj\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:10:57.798384 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.798333 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:57.798765 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.798752 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="135c218b-48cc-4e67-8d62-2de1e7999c7e" containerName="test-curl-7638-139999459902640" Jul 20 16:10:57.798765 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.798780 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="135c218b-48cc-4e67-8d62-2de1e7999c7e" containerName="test-curl-7638-139999459902640" Jul 20 16:10:57.798879 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.798839 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="135c218b-48cc-4e67-8d62-2de1e7999c7e" containerName="test-curl-7638-139999459902640" Jul 20 16:10:57.801638 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.801619 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:57.812212 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.812188 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:57.843359 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.843327 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfmfv\" (UniqueName: \"kubernetes.io/projected/820d676b-bd31-4d2c-9d84-dc26d2e4516f-kube-api-access-dfmfv\") pod \"test-curl-7638-139999459902640\" (UID: \"820d676b-bd31-4d2c-9d84-dc26d2e4516f\") " pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:57.944253 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.944212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfmfv\" (UniqueName: \"kubernetes.io/projected/820d676b-bd31-4d2c-9d84-dc26d2e4516f-kube-api-access-dfmfv\") pod \"test-curl-7638-139999459902640\" (UID: \"820d676b-bd31-4d2c-9d84-dc26d2e4516f\") " pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:57.956396 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:57.956365 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfmfv\" (UniqueName: \"kubernetes.io/projected/820d676b-bd31-4d2c-9d84-dc26d2e4516f-kube-api-access-dfmfv\") pod \"test-curl-7638-139999459902640\" (UID: \"820d676b-bd31-4d2c-9d84-dc26d2e4516f\") " pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:58.111313 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.111208 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:58.251947 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.251853 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:10:58.254542 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:10:58.254514 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod820d676b_bd31_4d2c_9d84_dc26d2e4516f.slice/crio-f9c5e9f51aeccec4fce3f080cf7b9731ac6f46e094a97f85c5cc0f08c6c56e9c WatchSource:0}: Error finding container f9c5e9f51aeccec4fce3f080cf7b9731ac6f46e094a97f85c5cc0f08c6c56e9c: Status 404 returned error can't find the container with id f9c5e9f51aeccec4fce3f080cf7b9731ac6f46e094a97f85c5cc0f08c6c56e9c Jul 20 16:10:58.457657 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.457618 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459902640" event={"ID":"820d676b-bd31-4d2c-9d84-dc26d2e4516f","Type":"ContainerStarted","Data":"f9c5e9f51aeccec4fce3f080cf7b9731ac6f46e094a97f85c5cc0f08c6c56e9c"} Jul 20 16:10:58.458719 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.458692 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:10:58.458826 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.458723 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="424cf443d36e45a80ccbb7b557ad43c6b7b8ef480398f256fbfaf74bcba75b16" Jul 20 16:10:58.472181 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.472152 2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7638-139999459902640" oldPodUID="135c218b-48cc-4e67-8d62-2de1e7999c7e" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" Jul 20 16:10:58.568041 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:58.568012 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="135c218b-48cc-4e67-8d62-2de1e7999c7e" path="/var/lib/kubelet/pods/135c218b-48cc-4e67-8d62-2de1e7999c7e/volumes" Jul 20 16:10:59.465954 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:59.465918 2557 generic.go:356] "Generic (PLEG): container finished" podID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" containerID="30ad07951bc3fb8f8337b75c35d97b5f74f6432dddb30b074c2dfe0ea2b5b648" exitCode=0 Jul 20 16:10:59.466354 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:10:59.465979 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459902640" event={"ID":"820d676b-bd31-4d2c-9d84-dc26d2e4516f","Type":"ContainerDied","Data":"30ad07951bc3fb8f8337b75c35d97b5f74f6432dddb30b074c2dfe0ea2b5b648"} Jul 20 16:11:00.608448 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.608420 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:11:00.642831 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.642790 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7638-139999459902640_820d676b-bd31-4d2c-9d84-dc26d2e4516f/test-curl-7638-139999459902640/0.log" Jul 20 16:11:00.670202 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.670166 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfmfv\" (UniqueName: \"kubernetes.io/projected/820d676b-bd31-4d2c-9d84-dc26d2e4516f-kube-api-access-dfmfv\") pod \"820d676b-bd31-4d2c-9d84-dc26d2e4516f\" (UID: \"820d676b-bd31-4d2c-9d84-dc26d2e4516f\") " Jul 20 16:11:00.672444 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.672416 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/820d676b-bd31-4d2c-9d84-dc26d2e4516f-kube-api-access-dfmfv" (OuterVolumeSpecName: "kube-api-access-dfmfv") pod "820d676b-bd31-4d2c-9d84-dc26d2e4516f" (UID: "820d676b-bd31-4d2c-9d84-dc26d2e4516f"). InnerVolumeSpecName "kube-api-access-dfmfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:11:00.688579 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.688555 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:11:00.698556 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.698531 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7638-139999459902640"] Jul 20 16:11:00.771852 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.771739 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfmfv\" (UniqueName: \"kubernetes.io/projected/820d676b-bd31-4d2c-9d84-dc26d2e4516f-kube-api-access-dfmfv\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:00.831360 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.831326 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:00.831799 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.831762 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" containerName="test-curl-7638-139999459902640" Jul 20 16:11:00.831799 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.831799 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" containerName="test-curl-7638-139999459902640" Jul 20 16:11:00.831912 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.831888 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" containerName="test-curl-7638-139999459902640" Jul 20 16:11:00.834947 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.834932 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:00.843033 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.843010 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:00.872229 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.872200 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtjpl\" (UniqueName: \"kubernetes.io/projected/b2e2a45c-f627-41eb-a7d8-14c346176a45-kube-api-access-qtjpl\") pod \"test-curl-7638-139999459903648\" (UID: \"b2e2a45c-f627-41eb-a7d8-14c346176a45\") " pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:00.972979 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.972942 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtjpl\" (UniqueName: \"kubernetes.io/projected/b2e2a45c-f627-41eb-a7d8-14c346176a45-kube-api-access-qtjpl\") pod \"test-curl-7638-139999459903648\" (UID: \"b2e2a45c-f627-41eb-a7d8-14c346176a45\") " pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:00.982065 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:00.982034 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtjpl\" (UniqueName: \"kubernetes.io/projected/b2e2a45c-f627-41eb-a7d8-14c346176a45-kube-api-access-qtjpl\") pod \"test-curl-7638-139999459903648\" (UID: \"b2e2a45c-f627-41eb-a7d8-14c346176a45\") " pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:01.145810 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:01.145762 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:01.284985 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:01.284959 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:01.286902 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:11:01.286873 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2e2a45c_f627_41eb_a7d8_14c346176a45.slice/crio-5f1a80d39bce6cbda533b6d79cc3a3e9177ffeeb9574fea6696dcaef28dca1eb WatchSource:0}: Error finding container 5f1a80d39bce6cbda533b6d79cc3a3e9177ffeeb9574fea6696dcaef28dca1eb: Status 404 returned error can't find the container with id 5f1a80d39bce6cbda533b6d79cc3a3e9177ffeeb9574fea6696dcaef28dca1eb Jul 20 16:11:01.478548 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:01.478455 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459903648" event={"ID":"b2e2a45c-f627-41eb-a7d8-14c346176a45","Type":"ContainerStarted","Data":"5f1a80d39bce6cbda533b6d79cc3a3e9177ffeeb9574fea6696dcaef28dca1eb"} Jul 20 16:11:01.479757 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:01.479736 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9c5e9f51aeccec4fce3f080cf7b9731ac6f46e094a97f85c5cc0f08c6c56e9c" Jul 20 16:11:01.479907 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:01.479756 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:11:01.492281 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:11:01.492251 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999459902640\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:11:02.487409 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:02.487374 2557 generic.go:356] "Generic (PLEG): container finished" podID="b2e2a45c-f627-41eb-a7d8-14c346176a45" containerID="e9fd127b9d7eb6b28b410a260e8ad20686ba265dc71dbab47b96d53c38ab7d36" exitCode=0 Jul 20 16:11:02.487823 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:02.487465 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459903648" event={"ID":"b2e2a45c-f627-41eb-a7d8-14c346176a45","Type":"ContainerDied","Data":"e9fd127b9d7eb6b28b410a260e8ad20686ba265dc71dbab47b96d53c38ab7d36"} Jul 20 16:11:02.518450 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:11:02.518402 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999459902640\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" pod="opendatahub/test-curl-7638-139999459902640" Jul 20 16:11:02.566873 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:02.566839 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="820d676b-bd31-4d2c-9d84-dc26d2e4516f" path="/var/lib/kubelet/pods/820d676b-bd31-4d2c-9d84-dc26d2e4516f/volumes" Jul 20 16:11:03.635368 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.635344 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:03.656964 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.656936 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7638-139999459903648_b2e2a45c-f627-41eb-a7d8-14c346176a45/test-curl-7638-139999459903648/0.log" Jul 20 16:11:03.685478 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.685447 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:03.689166 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.689142 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:03.701021 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.700998 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtjpl\" (UniqueName: \"kubernetes.io/projected/b2e2a45c-f627-41eb-a7d8-14c346176a45-kube-api-access-qtjpl\") pod \"b2e2a45c-f627-41eb-a7d8-14c346176a45\" (UID: \"b2e2a45c-f627-41eb-a7d8-14c346176a45\") " Jul 20 16:11:03.703172 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.703151 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e2a45c-f627-41eb-a7d8-14c346176a45-kube-api-access-qtjpl" (OuterVolumeSpecName: "kube-api-access-qtjpl") pod "b2e2a45c-f627-41eb-a7d8-14c346176a45" (UID: "b2e2a45c-f627-41eb-a7d8-14c346176a45"). InnerVolumeSpecName "kube-api-access-qtjpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:11:03.801813 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.801703 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtjpl\" (UniqueName: \"kubernetes.io/projected/b2e2a45c-f627-41eb-a7d8-14c346176a45-kube-api-access-qtjpl\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:03.820475 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.820442 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:03.820913 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.820894 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e2a45c-f627-41eb-a7d8-14c346176a45" containerName="test-curl-7638-139999459903648" Jul 20 16:11:03.820993 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.820915 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e2a45c-f627-41eb-a7d8-14c346176a45" containerName="test-curl-7638-139999459903648" Jul 20 16:11:03.821055 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.821044 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e2a45c-f627-41eb-a7d8-14c346176a45" containerName="test-curl-7638-139999459903648" Jul 20 16:11:03.824011 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.823993 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:03.829307 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.829284 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:03.902362 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:03.902323 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlp92\" (UniqueName: \"kubernetes.io/projected/24d9de95-9c50-4431-be77-a1a04ba11f28-kube-api-access-zlp92\") pod \"test-curl-7638-139999459903648\" (UID: \"24d9de95-9c50-4431-be77-a1a04ba11f28\") " pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:04.003883 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.003847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlp92\" (UniqueName: \"kubernetes.io/projected/24d9de95-9c50-4431-be77-a1a04ba11f28-kube-api-access-zlp92\") pod \"test-curl-7638-139999459903648\" (UID: \"24d9de95-9c50-4431-be77-a1a04ba11f28\") " pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:04.012259 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.012229 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlp92\" (UniqueName: \"kubernetes.io/projected/24d9de95-9c50-4431-be77-a1a04ba11f28-kube-api-access-zlp92\") pod \"test-curl-7638-139999459903648\" (UID: \"24d9de95-9c50-4431-be77-a1a04ba11f28\") " pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:04.138550 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.138521 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:04.278413 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.278388 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:04.279230 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:11:04.279209 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24d9de95_9c50_4431_be77_a1a04ba11f28.slice/crio-cd6f48bc9f001ee1ab13dc266b8f127636bfa23a8b756a10d36d45c7c57002a1 WatchSource:0}: Error finding container cd6f48bc9f001ee1ab13dc266b8f127636bfa23a8b756a10d36d45c7c57002a1: Status 404 returned error can't find the container with id cd6f48bc9f001ee1ab13dc266b8f127636bfa23a8b756a10d36d45c7c57002a1 Jul 20 16:11:04.500241 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.500159 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459903648" event={"ID":"24d9de95-9c50-4431-be77-a1a04ba11f28","Type":"ContainerStarted","Data":"cd6f48bc9f001ee1ab13dc266b8f127636bfa23a8b756a10d36d45c7c57002a1"} Jul 20 16:11:04.501407 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.501380 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f1a80d39bce6cbda533b6d79cc3a3e9177ffeeb9574fea6696dcaef28dca1eb" Jul 20 16:11:04.501507 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.501430 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:04.516200 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.516179 2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7638-139999459903648" oldPodUID="b2e2a45c-f627-41eb-a7d8-14c346176a45" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" Jul 20 16:11:04.566241 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:04.566216 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2e2a45c-f627-41eb-a7d8-14c346176a45" path="/var/lib/kubelet/pods/b2e2a45c-f627-41eb-a7d8-14c346176a45/volumes" Jul 20 16:11:05.509507 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:05.509472 2557 generic.go:356] "Generic (PLEG): container finished" podID="24d9de95-9c50-4431-be77-a1a04ba11f28" containerID="a458a869522a8943308244a57f6b6287b07aed58a611656bd963f86157b313ce" exitCode=0 Jul 20 16:11:05.509928 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:05.509556 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7638-139999459903648" event={"ID":"24d9de95-9c50-4431-be77-a1a04ba11f28","Type":"ContainerDied","Data":"a458a869522a8943308244a57f6b6287b07aed58a611656bd963f86157b313ce"} Jul 20 16:11:06.651642 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.651615 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:06.692000 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.691971 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7638-139999459903648_24d9de95-9c50-4431-be77-a1a04ba11f28/test-curl-7638-139999459903648/0.log" Jul 20 16:11:06.729118 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.729095 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlp92\" (UniqueName: \"kubernetes.io/projected/24d9de95-9c50-4431-be77-a1a04ba11f28-kube-api-access-zlp92\") pod \"24d9de95-9c50-4431-be77-a1a04ba11f28\" (UID: \"24d9de95-9c50-4431-be77-a1a04ba11f28\") " Jul 20 16:11:06.731591 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.731567 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d9de95-9c50-4431-be77-a1a04ba11f28-kube-api-access-zlp92" (OuterVolumeSpecName: "kube-api-access-zlp92") pod "24d9de95-9c50-4431-be77-a1a04ba11f28" (UID: "24d9de95-9c50-4431-be77-a1a04ba11f28"). InnerVolumeSpecName "kube-api-access-zlp92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:11:06.733602 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.733583 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:06.751009 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.750985 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7638-139999459903648"] Jul 20 16:11:06.830297 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:06.830242 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlp92\" (UniqueName: \"kubernetes.io/projected/24d9de95-9c50-4431-be77-a1a04ba11f28-kube-api-access-zlp92\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:07.523590 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:07.523560 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:07.523813 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:07.523600 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd6f48bc9f001ee1ab13dc266b8f127636bfa23a8b756a10d36d45c7c57002a1" Jul 20 16:11:07.537411 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:11:07.537377 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999459903648\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:08.564387 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:11:08.564347 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7638-139999459903648\" is forbidden: User \"system:node:ip-10-0-141-150.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-150.ec2.internal' and this object" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" pod="opendatahub/test-curl-7638-139999459903648" Jul 20 16:11:08.565873 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:08.565855 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" path="/var/lib/kubelet/pods/24d9de95-9c50-4431-be77-a1a04ba11f28/volumes" Jul 20 16:11:20.681001 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:20.680967 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:11:30.170262 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:30.170225 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr"] Jul 20 16:11:30.170898 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:30.170439 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" containerID="cri-o://cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11" gracePeriod=30 Jul 20 16:11:35.417687 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.417663 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:11:35.485455 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485382 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-podinfo\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485455 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485435 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-envoy\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485455 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485455 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-socket\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485658 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485574 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e03a4a-861e-435a-b2ac-0339b1f352fa-istiod-ca-cert\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485702 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485680 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-certs\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485752 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485697 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:11:35.485752 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485713 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmhd6\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-kube-api-access-dmhd6\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485752 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485747 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-data\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485938 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485799 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-token\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485938 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485858 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-credential-socket\") pod \"55e03a4a-861e-435a-b2ac-0339b1f352fa\" (UID: \"55e03a4a-861e-435a-b2ac-0339b1f352fa\") " Jul 20 16:11:35.485938 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485914 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:11:35.486096 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.485966 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55e03a4a-861e-435a-b2ac-0339b1f352fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:11:35.486379 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.486356 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-certs\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.486470 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.486382 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-workload-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.486470 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.486398 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e03a4a-861e-435a-b2ac-0339b1f352fa-istiod-ca-cert\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.486470 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.486411 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:11:35.486803 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.486757 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:11:35.487765 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.487743 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 16:11:35.487765 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.487753 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:11:35.488347 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.488319 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-kube-api-access-dmhd6" (OuterVolumeSpecName: "kube-api-access-dmhd6") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "kube-api-access-dmhd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:11:35.488347 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.488331 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "55e03a4a-861e-435a-b2ac-0339b1f352fa" (UID: "55e03a4a-861e-435a-b2ac-0339b1f352fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:11:35.587225 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.587199 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-envoy\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.587225 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.587223 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmhd6\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-kube-api-access-dmhd6\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.587376 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.587232 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-data\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.587376 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.587241 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-token\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.587376 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.587249 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e03a4a-861e-435a-b2ac-0339b1f352fa-credential-socket\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.587376 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.587256 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e03a4a-861e-435a-b2ac-0339b1f352fa-istio-podinfo\") on node \"ip-10-0-141-150.ec2.internal\" DevicePath \"\"" Jul 20 16:11:35.687582 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.687550 2557 generic.go:356] "Generic (PLEG): container finished" podID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerID="cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11" exitCode=0 Jul 20 16:11:35.687733 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.687598 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" Jul 20 16:11:35.687733 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.687634 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" event={"ID":"55e03a4a-861e-435a-b2ac-0339b1f352fa","Type":"ContainerDied","Data":"cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11"} Jul 20 16:11:35.687733 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.687696 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr" event={"ID":"55e03a4a-861e-435a-b2ac-0339b1f352fa","Type":"ContainerDied","Data":"4a348fa161646e947e2f6141f9b1ee9a4686139cc26daf0b996f0480511c30c0"} Jul 20 16:11:35.687733 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.687726 2557 scope.go:117] "RemoveContainer" containerID="cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11" Jul 20 16:11:35.698059 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.698043 2557 scope.go:117] "RemoveContainer" containerID="a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127" Jul 20 16:11:35.706548 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.706532 2557 scope.go:117] "RemoveContainer" containerID="cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11" Jul 20 16:11:35.706918 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:11:35.706892 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11\": container with ID starting with cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11 not found: ID does not exist" containerID="cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11" Jul 20 16:11:35.707056 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.706924 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11"} err="failed to get container status \"cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11\": rpc error: code = NotFound desc = could not find container \"cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11\": container with ID starting with cc059f5a80fea6e22dab8ba25292af28ec893ab902da906f90e1d479bb7f3c11 not found: ID does not exist" Jul 20 16:11:35.707056 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.706949 2557 scope.go:117] "RemoveContainer" containerID="a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127" Jul 20 16:11:35.707370 ip-10-0-141-150 kubenswrapper[2557]: E0720 16:11:35.707338 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127\": container with ID starting with a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127 not found: ID does not exist" containerID="a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127" Jul 20 16:11:35.707482 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.707378 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127"} err="failed to get container status \"a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127\": rpc error: code = NotFound desc = could not find container \"a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127\": container with ID starting with a983922b48943d3387f71b45a849f5d5235ea3af91e517de2e79d9cb8e3bb127 not found: ID does not exist" Jul 20 16:11:35.709896 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.709875 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr"] Jul 20 16:11:35.717977 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:35.717957 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8791e0-openshift-default-698c5f5867-22hjr"] Jul 20 16:11:36.566712 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:36.566677 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" path="/var/lib/kubelet/pods/55e03a4a-861e-435a-b2ac-0339b1f352fa/volumes" Jul 20 16:11:43.886663 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:43.886582 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-br5t2"] Jul 20 16:11:48.935986 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:48.935962 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 16:11:48.952455 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:48.952431 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6hzlp_c9a8cd6d-0178-49fb-b191-0e7d148f7ea9/ovn-acl-logging/0.log" Jul 20 16:11:58.998224 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:58.998190 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dxglr_cc609f54-fe6f-49d6-8239-85efc7351571/manager/0.log" Jul 20 16:11:59.361409 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:59.361324 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zsftq_36976c57-7182-4cac-bba4-7bbd44759099/server/0.log" Jul 20 16:11:59.602449 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:59.602424 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-d9f67_7b4e6a08-1230-4abc-bf14-1d91f5f51040/copy-manifests/0.log" Jul 20 16:11:59.610490 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:11:59.610468 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-d9f67_7b4e6a08-1230-4abc-bf14-1d91f5f51040/manager/0.log" Jul 20 16:12:01.390962 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.390930 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn_bbadba61-e355-4b28-9725-a9b76ff809f7/util/0.log" Jul 20 16:12:01.398214 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.398178 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn_bbadba61-e355-4b28-9725-a9b76ff809f7/pull/0.log" Jul 20 16:12:01.405598 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.405571 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn_bbadba61-e355-4b28-9725-a9b76ff809f7/extract/0.log" Jul 20 16:12:01.516605 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.516574 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5_df101e46-4c7d-4942-864a-4587988ed46d/util/0.log" Jul 20 16:12:01.523595 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.523561 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5_df101e46-4c7d-4942-864a-4587988ed46d/pull/0.log" Jul 20 16:12:01.530060 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.530041 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5_df101e46-4c7d-4942-864a-4587988ed46d/extract/0.log" Jul 20 16:12:01.647933 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.647852 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l_be2760a3-1789-42bc-9f82-41044e67d6e7/pull/0.log" Jul 20 16:12:01.655695 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.655657 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l_be2760a3-1789-42bc-9f82-41044e67d6e7/extract/0.log" Jul 20 16:12:01.663243 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.663224 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l_be2760a3-1789-42bc-9f82-41044e67d6e7/util/0.log" Jul 20 16:12:01.780269 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.780241 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz_99cfb23c-0d5d-4ffe-bb23-4d81dc85213e/util/0.log" Jul 20 16:12:01.789960 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.789936 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz_99cfb23c-0d5d-4ffe-bb23-4d81dc85213e/pull/0.log" Jul 20 16:12:01.798159 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:01.798140 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz_99cfb23c-0d5d-4ffe-bb23-4d81dc85213e/extract/0.log" Jul 20 16:12:02.051017 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.050926 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-q4htx_413ad218-87ae-485e-bf0a-4a3ad7f2b07c/manager/0.log" Jul 20 16:12:02.167857 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.167825 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-srrvm_ea3a7ddc-69c0-4035-802f-eb48d1135353/manager/0.log" Jul 20 16:12:02.281683 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.281658 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xg8mb_e571d3cd-7d57-4742-b0b6-1949be6e4496/kuadrant-console-plugin/0.log" Jul 20 16:12:02.399133 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.399108 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-f4nwv_91ac4863-7768-4482-843d-f8a2f3b76f75/registry-server/0.log" Jul 20 16:12:02.524613 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.524588 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x95zd_4dd01d83-693f-47e1-a193-8d50af239cda/manager/0.log" Jul 20 16:12:02.644469 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.644446 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-br5t2_97641873-5e39-44c8-ab98-211a181c8d49/limitador/0.log" Jul 20 16:12:02.766687 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:02.766626 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-znpxx_5ba8fb3b-925d-498d-9bbb-0e18a7541bc3/manager/0.log" Jul 20 16:12:03.125035 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:03.125009 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gm7pv_403a8ea7-a880-4ecb-bf88-1b36beef776e/istio-proxy/0.log" Jul 20 16:12:03.469402 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:03.469323 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-ltf4s_a557493c-e288-4800-9557-e5e63567342e/kube-auth-proxy/0.log" Jul 20 16:12:03.600304 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:03.600267 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/1.log" Jul 20 16:12:04.649859 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:04.649824 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhbtd2_3582e79a-91a3-4387-a0a7-3c923ae3135a/main/0.log" Jul 20 16:12:04.878990 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:04.878945 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-nr9zz_a4c35af4-9e30-474b-9090-7d28319f78e0/main/0.log" Jul 20 16:12:13.433435 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:13.433403 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ddwq7_598ac123-40fd-474c-b70e-8877d0aef642/global-pull-secret-syncer/0.log" Jul 20 16:12:13.740657 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:13.740576 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kq5qm_2a13d15d-b2df-4eca-9a3f-f4ebdc229404/konnectivity-agent/0.log" Jul 20 16:12:13.816983 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:13.816958 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-150.ec2.internal_568e315d1498f6b05c03ff2bf976f5b2/haproxy/0.log" Jul 20 16:12:17.831785 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:17.831737 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn_bbadba61-e355-4b28-9725-a9b76ff809f7/extract/0.log" Jul 20 16:12:17.859853 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:17.859827 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn_bbadba61-e355-4b28-9725-a9b76ff809f7/util/0.log" Jul 20 16:12:17.888269 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:17.888231 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kmlvn_bbadba61-e355-4b28-9725-a9b76ff809f7/pull/0.log" Jul 20 16:12:17.935580 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:17.935545 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5_df101e46-4c7d-4942-864a-4587988ed46d/extract/0.log" Jul 20 16:12:17.961845 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:17.961824 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5_df101e46-4c7d-4942-864a-4587988ed46d/util/0.log" Jul 20 16:12:17.987024 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:17.987002 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08rkt5_df101e46-4c7d-4942-864a-4587988ed46d/pull/0.log" Jul 20 16:12:18.016162 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.016136 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l_be2760a3-1789-42bc-9f82-41044e67d6e7/extract/0.log" Jul 20 16:12:18.041548 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.041527 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l_be2760a3-1789-42bc-9f82-41044e67d6e7/util/0.log" Jul 20 16:12:18.066367 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.066347 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xs22l_be2760a3-1789-42bc-9f82-41044e67d6e7/pull/0.log" Jul 20 16:12:18.096456 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.096392 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz_99cfb23c-0d5d-4ffe-bb23-4d81dc85213e/extract/0.log" Jul 20 16:12:18.120130 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.120109 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz_99cfb23c-0d5d-4ffe-bb23-4d81dc85213e/util/0.log" Jul 20 16:12:18.143684 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.143666 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brcmz_99cfb23c-0d5d-4ffe-bb23-4d81dc85213e/pull/0.log" Jul 20 16:12:18.640704 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.640668 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-q4htx_413ad218-87ae-485e-bf0a-4a3ad7f2b07c/manager/0.log" Jul 20 16:12:18.665541 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.665509 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-srrvm_ea3a7ddc-69c0-4035-802f-eb48d1135353/manager/0.log" Jul 20 16:12:18.702984 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.702957 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xg8mb_e571d3cd-7d57-4742-b0b6-1949be6e4496/kuadrant-console-plugin/0.log" Jul 20 16:12:18.764876 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.764848 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-f4nwv_91ac4863-7768-4482-843d-f8a2f3b76f75/registry-server/0.log" Jul 20 16:12:18.939379 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.939295 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x95zd_4dd01d83-693f-47e1-a193-8d50af239cda/manager/0.log" Jul 20 16:12:18.966589 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:18.966563 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-br5t2_97641873-5e39-44c8-ab98-211a181c8d49/limitador/0.log" Jul 20 16:12:19.067264 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:19.067225 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-znpxx_5ba8fb3b-925d-498d-9bbb-0e18a7541bc3/manager/0.log" Jul 20 16:12:20.673018 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.672986 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/alertmanager/0.log" Jul 20 16:12:20.702360 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.702340 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/config-reloader/0.log" Jul 20 16:12:20.728838 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.728811 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/kube-rbac-proxy-web/0.log" Jul 20 16:12:20.758759 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.758738 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/kube-rbac-proxy/0.log" Jul 20 16:12:20.788658 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.788635 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/kube-rbac-proxy-metric/0.log" Jul 20 16:12:20.812469 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.812446 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/prom-label-proxy/0.log" Jul 20 16:12:20.835674 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:20.835655 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_979890d9-4a3c-4257-be1e-4a26f6b91033/init-config-reloader/0.log" Jul 20 16:12:21.014088 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.014067 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b5cd9d9f4-bgv97_9d1c314e-ac25-4670-b116-1c703d9181cd/metrics-server/0.log" Jul 20 16:12:21.280942 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.280855 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srr59_bc87ef23-5059-45dc-8451-f31e41dd50f0/node-exporter/0.log" Jul 20 16:12:21.303148 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.303117 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srr59_bc87ef23-5059-45dc-8451-f31e41dd50f0/kube-rbac-proxy/0.log" Jul 20 16:12:21.330165 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.330140 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srr59_bc87ef23-5059-45dc-8451-f31e41dd50f0/init-textfile/0.log" Jul 20 16:12:21.360220 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.360193 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zcw8l_9e05225f-7edc-4014-a59c-a094a6de013d/kube-rbac-proxy-main/0.log" Jul 20 16:12:21.385108 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.385078 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zcw8l_9e05225f-7edc-4014-a59c-a094a6de013d/kube-rbac-proxy-self/0.log" Jul 20 16:12:21.409320 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.409296 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zcw8l_9e05225f-7edc-4014-a59c-a094a6de013d/openshift-state-metrics/0.log" Jul 20 16:12:21.658163 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.658133 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-hp9dh_cca73d00-0744-4541-80ae-e58215610fc3/prometheus-operator/0.log" Jul 20 16:12:21.683656 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.683630 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-hp9dh_cca73d00-0744-4541-80ae-e58215610fc3/kube-rbac-proxy/0.log" Jul 20 16:12:21.718230 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.718191 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-grmsw_6f9adadb-cd8d-404b-9f8a-4eddbe6b98c2/prometheus-operator-admission-webhook/0.log" Jul 20 16:12:21.751305 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.751264 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f499b659c-7p4tj_343c9bf5-baf3-4b4c-9255-8ddc7c560026/telemeter-client/0.log" Jul 20 16:12:21.778252 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.778231 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f499b659c-7p4tj_343c9bf5-baf3-4b4c-9255-8ddc7c560026/reload/0.log" Jul 20 16:12:21.801894 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:21.801873 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f499b659c-7p4tj_343c9bf5-baf3-4b4c-9255-8ddc7c560026/kube-rbac-proxy/0.log" Jul 20 16:12:23.316437 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.316396 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5"] Jul 20 16:12:23.316994 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.316974 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" Jul 20 16:12:23.317074 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.316997 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" Jul 20 16:12:23.317074 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.317011 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" containerName="test-curl-7638-139999459903648" Jul 20 16:12:23.317074 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.317019 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" containerName="test-curl-7638-139999459903648" Jul 20 16:12:23.317279 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.317141 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="55e03a4a-861e-435a-b2ac-0339b1f352fa" containerName="istio-proxy" Jul 20 16:12:23.317279 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.317157 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d9de95-9c50-4431-be77-a1a04ba11f28" containerName="test-curl-7638-139999459903648" Jul 20 16:12:23.320654 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.320632 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.322958 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.322933 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xwbnm\"/\"openshift-service-ca.crt\"" Jul 20 16:12:23.323241 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.323224 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xwbnm\"/\"default-dockercfg-j8nw4\"" Jul 20 16:12:23.323390 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.323373 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xwbnm\"/\"kube-root-ca.crt\"" Jul 20 16:12:23.333968 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.333947 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5"] Jul 20 16:12:23.408027 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.407999 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-sys\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.408182 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.408067 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-podres\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.408182 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.408090 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9llt\" (UniqueName: \"kubernetes.io/projected/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-kube-api-access-w9llt\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.408182 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.408130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-proc\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.408182 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.408173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-lib-modules\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509197 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-sys\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509372 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509234 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-podres\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509372 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9llt\" (UniqueName: \"kubernetes.io/projected/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-kube-api-access-w9llt\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509372 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-proc\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509372 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509310 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-lib-modules\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509372 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-sys\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509569 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509381 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-podres\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509569 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509398 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-proc\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.509569 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.509423 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-lib-modules\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.517597 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.517570 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9llt\" (UniqueName: \"kubernetes.io/projected/aa133276-7ec6-487e-b9d8-8a20a2cd99a1-kube-api-access-w9llt\") pod \"perf-node-gather-daemonset-jm4x5\" (UID: \"aa133276-7ec6-487e-b9d8-8a20a2cd99a1\") " pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.630141 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.630114 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.768054 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.768027 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5"] Jul 20 16:12:23.770266 ip-10-0-141-150 kubenswrapper[2557]: W0720 16:12:23.770239 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaa133276_7ec6_487e_b9d8_8a20a2cd99a1.slice/crio-38944c7b21b51e4430a948f3828d8ef215f0533178f6ce47ba1f5ce7f5fdf45a WatchSource:0}: Error finding container 38944c7b21b51e4430a948f3828d8ef215f0533178f6ce47ba1f5ce7f5fdf45a: Status 404 returned error can't find the container with id 38944c7b21b51e4430a948f3828d8ef215f0533178f6ce47ba1f5ce7f5fdf45a Jul 20 16:12:23.972660 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.972571 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" event={"ID":"aa133276-7ec6-487e-b9d8-8a20a2cd99a1","Type":"ContainerStarted","Data":"5311f5879eb3c5fb01341d59ecb38af951ccd9a487b3a992bfb2dbac10c82ea9"} Jul 20 16:12:23.972660 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.972627 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" event={"ID":"aa133276-7ec6-487e-b9d8-8a20a2cd99a1","Type":"ContainerStarted","Data":"38944c7b21b51e4430a948f3828d8ef215f0533178f6ce47ba1f5ce7f5fdf45a"} Jul 20 16:12:23.972660 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.972656 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:23.989258 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:23.989212 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" podStartSLOduration=0.989197208 podStartE2EDuration="989.197208ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:12:23.987406304 +0000 UTC m=+2735.931999675" watchObservedRunningTime="2026-07-20 16:12:23.989197208 +0000 UTC m=+2735.933790577" Jul 20 16:12:24.334445 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:24.334369 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69dfb79d48-kpgvf_6b11b8be-4eba-4435-b215-5e84eb2d0e73/console/0.log" Jul 20 16:12:25.977574 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:25.977547 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2c5dx_df2e94b7-bd9a-479c-9472-a69cb1bf9e58/dns/0.log" Jul 20 16:12:26.002722 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:26.002700 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2c5dx_df2e94b7-bd9a-479c-9472-a69cb1bf9e58/kube-rbac-proxy/0.log" Jul 20 16:12:26.135204 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:26.135176 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zkpd4_326a71ed-13ac-428f-8784-72a0bfb9eff2/dns-node-resolver/0.log" Jul 20 16:12:26.855254 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:26.855222 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ntqhn_c42f52c3-250c-484e-a5bc-0b3c1c92f4dd/node-ca/0.log" Jul 20 16:12:27.838901 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:27.838865 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gm7pv_403a8ea7-a880-4ecb-bf88-1b36beef776e/istio-proxy/0.log" Jul 20 16:12:28.025851 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:28.025821 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-ltf4s_a557493c-e288-4800-9557-e5e63567342e/kube-auth-proxy/0.log" Jul 20 16:12:28.083061 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:28.083030 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/0.log" Jul 20 16:12:28.131921 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:28.131894 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/1.log" Jul 20 16:12:28.923833 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:28.923799 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lhfxq_5bfe5645-2124-423a-921e-f294097db8d9/serve-healthcheck-canary/0.log" Jul 20 16:12:29.637418 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:29.637385 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nghgd_493dd2d2-5ce0-417c-8598-444afa797b51/kube-rbac-proxy/0.log" Jul 20 16:12:29.660258 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:29.660235 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nghgd_493dd2d2-5ce0-417c-8598-444afa797b51/exporter/0.log" Jul 20 16:12:29.684239 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:29.684213 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nghgd_493dd2d2-5ce0-417c-8598-444afa797b51/extractor/0.log" Jul 20 16:12:29.989074 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:29.989000 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xwbnm/perf-node-gather-daemonset-jm4x5" Jul 20 16:12:31.828113 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:31.828081 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dxglr_cc609f54-fe6f-49d6-8239-85efc7351571/manager/0.log" Jul 20 16:12:32.271877 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:32.271846 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zsftq_36976c57-7182-4cac-bba4-7bbd44759099/server/0.log" Jul 20 16:12:32.385484 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:32.385384 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-d9f67_7b4e6a08-1230-4abc-bf14-1d91f5f51040/manager/0.log" Jul 20 16:12:32.409618 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:32.409596 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65bbbccfd9-d9f67_7b4e6a08-1230-4abc-bf14-1d91f5f51040/copy-manifests/0.log" Jul 20 16:12:33.976849 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:33.976817 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9psb4_6c3e8d87-817c-4fa4-bc0d-f790f11693f3/openshift-lws-operator/0.log" Jul 20 16:12:34.030184 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:34.030156 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gm7pv_403a8ea7-a880-4ecb-bf88-1b36beef776e/istio-proxy/0.log" Jul 20 16:12:34.192071 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:34.192035 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-ltf4s_a557493c-e288-4800-9557-e5e63567342e/kube-auth-proxy/0.log" Jul 20 16:12:34.230258 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:34.230171 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/0.log" Jul 20 16:12:34.277541 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:34.277509 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/1.log" Jul 20 16:12:35.550274 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:35.550244 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gm7pv_403a8ea7-a880-4ecb-bf88-1b36beef776e/istio-proxy/0.log" Jul 20 16:12:35.703036 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:35.703004 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6598cc5cf9-ltf4s_a557493c-e288-4800-9557-e5e63567342e/kube-auth-proxy/0.log" Jul 20 16:12:35.725827 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:35.725727 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/0.log" Jul 20 16:12:35.776283 ip-10-0-141-150 kubenswrapper[2557]: I0720 16:12:35.776250 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xqn2b_e31d0278-dbd4-491a-a7b5-10a1e7c53d3c/istio-proxy/1.log"