Jul 21 12:11:06.398727 ip-10-0-136-33 systemd[1]: Starting Kubernetes Kubelet... Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.791573 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797194 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797206 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797210 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797215 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797218 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797222 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797226 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:11:06.876681 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797229 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797232 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797235 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797237 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797241 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797244 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797247 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797251 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797253 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797258 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797263 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797268 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797271 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797274 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797278 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797281 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797284 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797287 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797290 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:11:06.878047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797292 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797295 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797298 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797308 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797311 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797315 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797319 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797323 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797326 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797329 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797332 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797335 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797339 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797341 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797344 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797347 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797350 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797353 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797355 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797358 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:11:06.879110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797361 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797363 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797366 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797369 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797373 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797376 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797379 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797382 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797385 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797388 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797391 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797394 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797397 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797399 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797402 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797405 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797408 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797411 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797413 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:11:06.881919 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797416 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797419 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797421 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797425 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797428 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797431 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797434 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797437 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797441 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797443 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797446 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797449 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797451 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797455 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797458 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797460 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797464 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797467 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797470 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797472 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:11:06.882848 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797475 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797478 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797481 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797483 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797486 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797489 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797492 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797494 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797512 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797517 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797522 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797526 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797529 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.797533 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798333 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798340 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798343 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798346 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798349 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798353 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:11:06.886021 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798357 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:11:06.886720 ip-10-0-136-33 systemd[1]: Started Kubernetes Kubelet. Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798359 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798362 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798365 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798368 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798371 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798373 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798376 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798379 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798382 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798385 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798388 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798390 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798393 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798396 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798399 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798401 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798404 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798407 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798410 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:11:06.887380 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798414 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798416 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798420 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798423 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798426 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798429 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798433 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798435 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798438 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798441 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798445 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798447 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798450 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798455 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798459 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798463 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798466 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798469 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798472 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:11:06.891464 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798474 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798477 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798480 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798486 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798489 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798492 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798495 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798516 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798520 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798524 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798529 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798533 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798536 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798539 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798542 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798544 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798547 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798551 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798553 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798556 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:11:06.892844 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798559 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798561 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798564 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798568 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798570 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798573 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798577 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798580 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798582 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798585 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798588 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798591 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798594 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798596 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798599 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798602 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798605 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798608 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798610 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798613 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:11:06.894011 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798616 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798618 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798621 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798624 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798627 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798631 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798635 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798638 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798642 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798644 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798647 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798650 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798653 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.798655 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799059 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799067 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799070 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799073 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799077 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:11:06.897289 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799079 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799082 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799085 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799088 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799091 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799094 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799096 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799099 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799103 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799105 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799108 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799111 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799115 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799119 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799122 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799124 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799127 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799130 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799133 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799136 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:11:06.898146 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799139 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799142 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799145 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799148 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799150 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799153 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799156 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799159 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799164 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799167 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799170 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799173 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799176 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799179 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799182 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799185 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799188 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799191 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799194 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799197 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:11:06.899314 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799200 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799202 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799205 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799208 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799210 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799213 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799216 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799218 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799221 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799224 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799227 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799230 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799233 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799236 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799239 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799241 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799244 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799247 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799250 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799252 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:11:06.901379 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799256 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799259 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799263 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799267 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799270 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799273 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799275 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799281 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799285 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799288 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799291 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799294 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799297 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799300 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799303 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799306 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799309 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799311 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:11:06.902539 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799314 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799318 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799322 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799325 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799328 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799331 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799334 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799337 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799339 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799342 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799344 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799347 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799350 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799352 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799357 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.799360 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799440 2550 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799455 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799461 2550 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799466 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799471 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 12:11:07.004373 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799474 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799479 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799484 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799487 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799490 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799494 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799516 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799522 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799527 2550 flags.go:64] FLAG: --cgroup-root="" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799532 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799537 2550 flags.go:64] FLAG: --client-ca-file="" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799540 2550 flags.go:64] FLAG: --cloud-provider="external" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799544 2550 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799549 2550 flags.go:64] FLAG: --cluster-domain="" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799552 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799556 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799559 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799563 2550 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799567 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799570 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799574 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799577 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799580 2550 flags.go:64] FLAG: --contention-profiling="false" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799583 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 12:11:07.005899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799587 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799591 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799595 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799599 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799604 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799607 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799610 2550 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799614 2550 flags.go:64] FLAG: --enable-server="true" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799617 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799622 2550 flags.go:64] FLAG: --event-burst="100" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799625 2550 flags.go:64] FLAG: --event-qps="50" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799628 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799631 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799634 2550 flags.go:64] FLAG: --eviction-hard="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799638 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799641 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799644 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799648 2550 flags.go:64] FLAG: --eviction-soft="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799651 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799654 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799657 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799660 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799663 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799666 2550 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799669 2550 flags.go:64] FLAG: --feature-gates="" Jul 21 12:11:07.009122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799673 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799676 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799679 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799683 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799686 2550 flags.go:64] FLAG: --healthz-port="10248" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799689 2550 flags.go:64] FLAG: --help="false" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799692 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799695 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799700 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799703 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799706 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799711 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799714 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799717 2550 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799720 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799729 2550 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799732 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799736 2550 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799739 2550 flags.go:64] FLAG: --kube-reserved="" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799742 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799745 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799748 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799751 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799754 2550 flags.go:64] FLAG: --lock-file="" Jul 21 12:11:07.010478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799757 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799760 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799763 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799769 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799772 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799775 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799778 2550 flags.go:64] FLAG: --logging-format="text" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799782 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799785 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799788 2550 flags.go:64] FLAG: --manifest-url="" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799791 2550 flags.go:64] FLAG: --manifest-url-header="" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799796 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799799 2550 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799803 2550 flags.go:64] FLAG: --max-pods="110" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799806 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799810 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799814 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799817 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799821 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799826 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799829 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799836 2550 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799840 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799843 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 12:11:07.156249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799846 2550 flags.go:64] FLAG: --pod-cidr="" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799850 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799856 2550 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799859 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799863 2550 flags.go:64] FLAG: --pods-per-core="0" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799866 2550 flags.go:64] FLAG: --port="10250" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799869 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799873 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b217bcf0061b2d58" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799876 2550 flags.go:64] FLAG: --qos-reserved="" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799879 2550 flags.go:64] FLAG: --read-only-port="10255" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799882 2550 flags.go:64] FLAG: --register-node="true" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799886 2550 flags.go:64] FLAG: --register-with-taints="" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799889 2550 flags.go:64] FLAG: --registry-burst="10" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799893 2550 flags.go:64] FLAG: --registry-qps="5" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799896 2550 flags.go:64] FLAG: --reserved-cpus="" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799899 2550 flags.go:64] FLAG: --reserved-memory="" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799903 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799906 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799909 2550 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799913 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799916 2550 flags.go:64] FLAG: --runonce="false" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799919 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799923 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799925 2550 flags.go:64] FLAG: --seccomp-default="false" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799928 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 12:11:07.180751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799932 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799936 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799941 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799944 2550 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799947 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799950 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799953 2550 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799956 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799960 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799964 2550 flags.go:64] FLAG: --system-cgroups="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799966 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799972 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799975 2550 flags.go:64] FLAG: --tls-cert-file="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799978 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799983 2550 flags.go:64] FLAG: --tls-min-version="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799986 2550 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799989 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799992 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799995 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.799998 2550 flags.go:64] FLAG: --v="2" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.800002 2550 flags.go:64] FLAG: --version="false" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.800006 2550 flags.go:64] FLAG: --vmodule="" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.800010 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.800014 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800141 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:11:07.181646 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800145 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800148 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800151 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800154 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800157 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800159 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800163 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800167 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800169 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800174 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800177 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800179 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800182 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800185 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800188 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800191 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800193 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800196 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800200 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800203 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:11:07.183155 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800206 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800209 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800211 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800214 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800217 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800219 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800222 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800225 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800227 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800230 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800234 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800237 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800239 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800242 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800245 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800248 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800251 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800254 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800256 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800260 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:11:07.183813 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800263 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800267 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800270 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800273 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800275 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800278 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800282 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800284 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800287 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800290 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800293 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800296 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800299 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800301 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800304 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800306 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800309 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800312 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800315 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:11:07.184486 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800318 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800320 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800325 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800328 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800332 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800335 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800338 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800342 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800346 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800349 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800352 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800355 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800360 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800363 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800368 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800371 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800373 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800376 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800379 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:11:07.185106 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800382 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800384 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800388 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800392 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800394 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800398 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800400 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800403 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800406 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800409 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800411 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800414 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800417 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800419 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800422 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800425 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800427 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800430 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800433 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:11:07.185720 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.800436 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.800442 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.811987 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.812004 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.812026 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.812035 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812100 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812108 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812114 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812117 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812120 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812123 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812126 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812130 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812135 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:11:07.186441 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812140 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812143 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812146 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812149 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812153 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812156 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812159 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812162 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812165 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812168 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812170 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812173 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812176 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812179 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812182 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812187 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812191 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812196 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812200 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812203 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:11:07.186940 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812207 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812210 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812213 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812216 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812218 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812221 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812225 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812227 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812230 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812233 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812235 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812238 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812241 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812244 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812246 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812249 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812252 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812255 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812258 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812260 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:11:07.187488 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812264 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812269 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812274 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812279 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812283 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812286 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812289 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812292 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812295 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812298 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812300 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812303 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812306 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812310 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812312 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812315 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812318 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812321 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812323 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:11:07.188047 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812328 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812331 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812335 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812337 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812340 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812343 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812346 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812349 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812354 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812358 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812363 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812366 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812369 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812372 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812375 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812378 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812380 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812383 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812386 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812389 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:11:07.188591 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812391 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812394 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812397 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812400 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812402 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812408 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812411 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812415 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812417 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812420 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812423 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.812429 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812614 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812623 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812627 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:11:07.189124 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812630 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812633 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812636 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812639 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812642 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812645 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812648 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812651 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812654 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812657 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812660 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812662 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812665 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812668 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812671 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812673 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812676 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812680 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812685 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:11:07.189608 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812690 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812696 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812700 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812706 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812709 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812711 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812715 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812718 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812721 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812725 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812727 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812730 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812732 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812735 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812738 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812741 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812743 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812746 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812749 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812751 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:11:07.190147 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812756 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812759 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812761 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812764 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812767 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812770 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812773 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812778 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812782 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812787 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812790 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812793 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812795 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812798 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812801 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812805 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812808 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812810 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812813 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:11:07.190704 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812817 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812820 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812822 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812825 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812828 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812831 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812834 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812838 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812842 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812846 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812849 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812852 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812855 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812861 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812867 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812871 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812874 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812877 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812880 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812882 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:11:07.191248 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812885 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812887 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812890 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812893 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812895 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812898 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812901 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812904 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812908 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812910 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812913 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812916 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812918 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812922 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812925 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812927 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812930 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:11:07.191798 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:06.812933 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.812937 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.813069 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.815449 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.816819 2550 server.go:1019] "Starting client certificate rotation" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.816930 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.818489 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.822621 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.826597 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.836367 2550 log.go:25] "Validated CRI v1 runtime API" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.838599 2550 log.go:25] "Validated CRI v1 image API" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.839441 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 12:11:07.192247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.849160 2550 fs.go:135] Filesystem UUIDs: map[4d2f3e8f-0b7d-40b5-b23a-0f230cef6260:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8b1db1f0-8105-4b4f-b0f2-5e30d60f159d:/dev/nvme0n1p3] Jul 21 12:11:07.192699 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.849178 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.854644 2550 manager.go:216] Machine: {Timestamp:2026-07-21 12:11:06.853184934 +0000 UTC m=+0.347013920 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108556 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb86e0d7b259f261fb5cbe4a919f5 SystemUUID:ec2eb86e-0d7b-259f-261f-b5cbe4a919f5 BootID:ce25168f-a3be-4d54-8a49-0c95268209b0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:eb:2f:c2:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:eb:2f:c2:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:09:30:9b:ae:8e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.854754 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.854842 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.858003 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.858035 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.858735 2550 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.858746 2550 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 12:11:07.192733 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.858756 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.859359 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.861058 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.865390 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.865567 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.866479 2550 kubelet.go:486] "Attempting to sync node with API server" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.866495 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.866526 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.866540 2550 kubelet.go:398] "Adding apiserver pod source" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.866555 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.867018 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.867039 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.869108 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.878714 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.878759 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.880280 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.880402 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880670 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880689 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880697 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880705 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880719 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880730 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880740 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880749 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880759 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880767 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880779 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.880788 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.882348 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.882360 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.885988 2550 server.go:1268] "Started kubelet" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.886397 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.888116 2550 server.go:310] "Adding debug handlers to kubelet server" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.889155 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.889195 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.891526 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.897703 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.897741 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 12:11:07.192994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.898194 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.898263 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-33.ec2.internal.18c44cdd9328b87f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-33.ec2.internal,UID:ip-10-0-136-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-33.ec2.internal,},FirstTimestamp:2026-07-21 12:11:06.885961855 +0000 UTC m=+0.379790842,LastTimestamp:2026-07-21 12:11:06.885961855 +0000 UTC m=+0.379790842,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-33.ec2.internal,}" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.901967 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.902946 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903181 2550 factory.go:55] Registering systemd factory Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903202 2550 factory.go:223] Registration of the systemd container factory successfully Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903476 2550 factory.go:153] Registering CRI-O factory Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903489 2550 factory.go:223] Registration of the crio container factory successfully Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903560 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903590 2550 factory.go:103] Registering Raw factory Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.903606 2550 manager.go:1196] Started watching for new ooms in manager Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.904472 2550 manager.go:349] Starting recovery of all containers Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.905227 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.905252 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.905351 2550 reconstruct.go:97] "Volume reconstruction finished" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.905362 2550 reconciler.go:29] "Reconciler: start to sync state" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.905454 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.908989 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.909085 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.919695 2550 manager.go:354] Recovery completed Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.928363 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.928538 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.930744 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.930780 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.194118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.930796 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.932792 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-33.ec2.internal.18c44cdd95d45023 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-33.ec2.internal,UID:ip-10-0-136-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-33.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-33.ec2.internal,},FirstTimestamp:2026-07-21 12:11:06.930761763 +0000 UTC m=+0.424590750,LastTimestamp:2026-07-21 12:11:06.930761763 +0000 UTC m=+0.424590750,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-33.ec2.internal,}" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.943519 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cx7gw" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.943725 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cx7gw" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.944010 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.944021 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.944026 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.944037 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.945155 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.945183 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.945194 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.945396 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-33.ec2.internal.18c44cdd95d4b8ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-33.ec2.internal,UID:ip-10-0-136-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-33.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-33.ec2.internal,},FirstTimestamp:2026-07-21 12:11:06.930788526 +0000 UTC m=+0.424617512,LastTimestamp:2026-07-21 12:11:06.930788526 +0000 UTC m=+0.424617512,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-33.ec2.internal,}" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.947067 2550 policy_none.go:49] "None policy: Start" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.947081 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.947091 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.949314 2550 policy_none.go:47] "Start" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999257 2550 manager.go:337] "Starting Device Plugin manager" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:06.999296 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999307 2550 server.go:87] "Starting device plugin registration server" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999655 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999666 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999773 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999874 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:06.999883 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.015809 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.015865 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.102224 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.194779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.103315 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.103348 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.103359 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.103389 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.111493 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.113170 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.113217 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.113242 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.113281 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.119036 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.119832 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.119852 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-33.ec2.internal\": node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.195571 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.160312 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.214032 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.213991 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal"] Jul 21 12:11:07.214166 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.214087 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.215224 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.215201 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.215313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.215244 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.215313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.215262 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.217967 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.217952 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.218075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218057 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.218121 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218092 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.218843 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218817 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.218979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218851 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.218979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218867 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.218979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218884 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.218979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218906 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.218979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.218919 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.221413 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.221392 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.221515 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.221429 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:11:07.222216 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.222199 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:11:07.222313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.222232 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:11:07.222313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.222244 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:11:07.253413 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.253389 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-33.ec2.internal\" not found" node="ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.258187 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.258166 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-33.ec2.internal\" not found" node="ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.261148 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.261130 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.307844 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.307817 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0a6cf30d9fe955de2f6516c2a1ccf33-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal\" (UID: \"e0a6cf30d9fe955de2f6516c2a1ccf33\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.307956 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.307851 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8dc81c72d91b2f4557f0246b9d50dc4e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-33.ec2.internal\" (UID: \"8dc81c72d91b2f4557f0246b9d50dc4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.307956 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.307869 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0a6cf30d9fe955de2f6516c2a1ccf33-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal\" (UID: \"e0a6cf30d9fe955de2f6516c2a1ccf33\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.363807 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.363773 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.408285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.408245 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0a6cf30d9fe955de2f6516c2a1ccf33-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal\" (UID: \"e0a6cf30d9fe955de2f6516c2a1ccf33\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.408285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.408287 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0a6cf30d9fe955de2f6516c2a1ccf33-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal\" (UID: \"e0a6cf30d9fe955de2f6516c2a1ccf33\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.408510 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.408311 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8dc81c72d91b2f4557f0246b9d50dc4e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-33.ec2.internal\" (UID: \"8dc81c72d91b2f4557f0246b9d50dc4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.408510 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.408345 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0a6cf30d9fe955de2f6516c2a1ccf33-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal\" (UID: \"e0a6cf30d9fe955de2f6516c2a1ccf33\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.408510 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.408369 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8dc81c72d91b2f4557f0246b9d50dc4e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-33.ec2.internal\" (UID: \"8dc81c72d91b2f4557f0246b9d50dc4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.408510 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.408403 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0a6cf30d9fe955de2f6516c2a1ccf33-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal\" (UID: \"e0a6cf30d9fe955de2f6516c2a1ccf33\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.467914 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.467871 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.558406 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.558325 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.561758 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.561736 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.568571 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.568550 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.671293 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.671250 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-33.ec2.internal\" not found" Jul 21 12:11:07.742306 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.742274 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:07.746348 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.746331 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:07.803957 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.803930 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.816617 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.816551 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 12:11:07.816714 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.816692 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:11:07.816766 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.816718 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:11:07.816766 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.816739 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:11:07.816844 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.816771 2550 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a68daf83198d14ae7a6c824cc97a8aaa-9da6bcf0860db144.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.136.33:45936->54.163.167.88:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.816844 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.816796 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" Jul 21 12:11:07.836931 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.836902 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:11:07.869567 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.869531 2550 apiserver.go:52] "Watching apiserver" Jul 21 12:11:07.878925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.878898 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 12:11:07.879347 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.879223 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-ks9gx","openshift-ovn-kubernetes/ovnkube-node-xwhch","kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg","openshift-cluster-node-tuning-operator/tuned-55hxx","openshift-dns/node-resolver-pb5qj","openshift-image-registry/node-ca-wxzfm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal","kube-system/global-pull-secret-syncer-rrb6l","kube-system/konnectivity-agent-m7bb6","openshift-multus/multus-additional-cni-plugins-7k4bk","openshift-multus/multus-bsc4n","openshift-multus/network-metrics-daemon-bgmwr","openshift-network-diagnostics/network-check-target-jknc7"] Jul 21 12:11:07.881982 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.881962 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:07.884600 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.884573 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t8dm4\"" Jul 21 12:11:07.884697 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.884655 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.884769 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.884715 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 12:11:07.884769 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.884735 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 12:11:07.887092 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887073 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:07.887190 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.887138 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:07.887256 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887200 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.887425 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887410 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 12:11:07.887519 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887425 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 12:11:07.887519 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887431 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 12:11:07.887519 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887497 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 12:11:07.887652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887601 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6b6b4\"" Jul 21 12:11:07.887652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.887618 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.889594 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.889578 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:07.891986 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.891969 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:07.892062 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.892037 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:07.894213 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.894194 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:07.896760 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.896741 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.897541 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.897506 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.897679 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.897663 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kl9lt\"" Jul 21 12:11:07.897926 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.897912 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.897992 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.897974 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 12:11:07.898050 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.898007 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cwkrs\"" Jul 21 12:11:07.898231 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.898214 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 12:11:07.898231 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.898228 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.899141 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.899126 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.901562 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.901538 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.902999 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.902977 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.903093 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903071 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 12:11:07.903164 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903123 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.903164 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903132 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 12:11:07.903287 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903270 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 12:11:07.903338 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903285 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j8dgf\"" Jul 21 12:11:07.903404 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903375 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 12:11:07.903489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903407 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.903545 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.903530 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z2tz5\"" Jul 21 12:11:07.904035 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.904019 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:07.904099 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:07.904072 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:07.904450 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.904436 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87qfx\"" Jul 21 12:11:07.904615 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.904594 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.904648 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.904628 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.907255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.905378 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 12:11:07.907847 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.907766 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:07.910299 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.910281 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:07.911690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911665 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7lm2\" (UniqueName: \"kubernetes.io/projected/91b0a1c8-c980-4d55-b819-a391a6a27fb7-kube-api-access-v7lm2\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.911775 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911708 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502316a9-ade5-4bf4-b1f8-f6868b87987b-hosts-file\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:07.911775 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911737 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-socket-dir-parent\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.911890 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911788 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-daemon-config\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.911890 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911827 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-systemd\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.911890 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911852 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovn-node-metrics-cert\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.911890 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911879 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-system-cni-dir\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911904 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-hostroot\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911928 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-var-lib-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911952 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.911977 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c40b72bd-871f-46a8-b390-2afe6322872f-host-slash\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912002 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912027 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4066c5a-25b5-4a23-80f7-ef95d3516d70-kubelet-config\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912067 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502316a9-ade5-4bf4-b1f8-f6868b87987b-tmp-dir\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:07.912126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912111 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-conf-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912180 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912187 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912178 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-kubelet\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912250 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-slash\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912275 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-cnibin\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912303 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-netns\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912328 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-cni-bin\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912339 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4572f\"" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912355 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-os-release\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912385 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-cni-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912413 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912413 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912443 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.912489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912492 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912521 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-device-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912546 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-cni-bin\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912588 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912631 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b8bfc2d-09ba-415b-82f2-bc35e42c93ff-agent-certs\") pod \"konnectivity-agent-m7bb6\" (UID: \"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff\") " pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912664 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-system-cni-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912703 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912718 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4066c5a-25b5-4a23-80f7-ef95d3516d70-dbus\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912764 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5db67\" (UniqueName: \"kubernetes.io/projected/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-kube-api-access-5db67\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912801 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-socket-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912842 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-k8s-cni-cncf-io\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912876 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c40b72bd-871f-46a8-b390-2afe6322872f-iptables-alerter-script\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912937 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:07.913067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.912961 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-etc-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913009 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cnibin\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913042 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhng6\" (UniqueName: \"kubernetes.io/projected/d58a9a1b-1177-4004-a9c0-c423078dace2-kube-api-access-rhng6\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913068 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b8bfc2d-09ba-415b-82f2-bc35e42c93ff-konnectivity-ca\") pod \"konnectivity-agent-m7bb6\" (UID: \"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff\") " pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913098 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcfx8\" (UniqueName: \"kubernetes.io/projected/5f84c1aa-7a4b-438c-84fb-47deb0a69877-kube-api-access-tcfx8\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913128 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppswd\" (UniqueName: \"kubernetes.io/projected/c40b72bd-871f-46a8-b390-2afe6322872f-kube-api-access-ppswd\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913153 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-sys-fs\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913177 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfw4d\" (UniqueName: \"kubernetes.io/projected/502316a9-ade5-4bf4-b1f8-f6868b87987b-kube-api-access-vfw4d\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913203 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f84c1aa-7a4b-438c-84fb-47deb0a69877-cni-binary-copy\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913238 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-multus-certs\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913271 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-systemd-units\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913296 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-env-overrides\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913321 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913348 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:07.913778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913364 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcwn9\"" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913372 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovnkube-script-lib\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913397 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-registration-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913408 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913475 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-os-release\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913499 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-node-log\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913540 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-log-socket\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913598 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-cni-netd\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913623 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovnkube-config\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913649 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913668 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913684 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-cni-multus\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913698 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-kubelet\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913714 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-etc-kubernetes\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:07.914190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913748 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-run-netns\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914607 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913778 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-ovn\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:07.914607 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.913802 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdddn\" (UniqueName: \"kubernetes.io/projected/a385e43d-14a9-4f43-a962-a08a8523cb5e-kube-api-access-bdddn\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:07.916191 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.916176 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:07.936737 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.936716 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:07.937392 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.937364 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:11:07.949276 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.949238 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:06:07 +0000 UTC" deadline="2028-02-06 03:21:42.230954112 +0000 UTC" Jul 21 12:11:07.949276 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.949275 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13551h10m34.281681271s" Jul 21 12:11:07.977721 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.977692 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ggms6" Jul 21 12:11:07.987162 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:07.987139 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ggms6" Jul 21 12:11:08.004570 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.004547 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 12:11:08.013916 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.013898 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfw4d\" (UniqueName: \"kubernetes.io/projected/502316a9-ade5-4bf4-b1f8-f6868b87987b-kube-api-access-vfw4d\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.013980 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.013925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f84c1aa-7a4b-438c-84fb-47deb0a69877-cni-binary-copy\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.013980 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.013943 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-multus-certs\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.013980 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.013958 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-systemd-units\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.013980 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.013974 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-env-overrides\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014013 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014037 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014039 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-multus-certs\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014055 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovnkube-script-lib\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014060 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-systemd-units\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014082 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-registration-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014128 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-os-release\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014164 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-node-log\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014170 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-registration-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.014131 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014224 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-log-socket\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014189 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-log-socket\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.014257 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:08.514236761 +0000 UTC m=+2.008065752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014272 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-node-log\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-cni-netd\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014342 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovnkube-config\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014398 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014572 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-env-overrides\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.014652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014634 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-cni-netd\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014674 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014225 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-os-release\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.014997 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-cni-multus\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015022 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovnkube-config\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015029 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-cni-multus\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015038 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-kubelet\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015076 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-etc-kubernetes\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015101 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-run-netns\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015115 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f84c1aa-7a4b-438c-84fb-47deb0a69877-cni-binary-copy\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015125 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-ovn\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015119 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-kubelet\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015151 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdddn\" (UniqueName: \"kubernetes.io/projected/a385e43d-14a9-4f43-a962-a08a8523cb5e-kube-api-access-bdddn\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015150 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovnkube-script-lib\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015138 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-etc-kubernetes\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.015226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015163 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015155 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-run-netns\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015192 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-ovn\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7lm2\" (UniqueName: \"kubernetes.io/projected/91b0a1c8-c980-4d55-b819-a391a6a27fb7-kube-api-access-v7lm2\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015255 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502316a9-ade5-4bf4-b1f8-f6868b87987b-hosts-file\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015281 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-socket-dir-parent\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015308 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-daemon-config\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015316 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502316a9-ade5-4bf4-b1f8-f6868b87987b-hosts-file\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015332 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-systemd\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015352 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovn-node-metrics-cert\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015375 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttq7j\" (UniqueName: \"kubernetes.io/projected/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-kube-api-access-ttq7j\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015384 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-systemd\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015350 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-socket-dir-parent\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015406 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-system-cni-dir\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015445 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-system-cni-dir\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.016056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015467 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-hostroot\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015486 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-var-lib-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015503 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015527 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysctl-conf\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015545 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-tmp\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015548 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-var-lib-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015555 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-run-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015562 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c40b72bd-871f-46a8-b390-2afe6322872f-host-slash\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015566 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-hostroot\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015594 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c40b72bd-871f-46a8-b390-2afe6322872f-host-slash\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015597 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015624 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4066c5a-25b5-4a23-80f7-ef95d3516d70-kubelet-config\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015649 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502316a9-ade5-4bf4-b1f8-f6868b87987b-tmp-dir\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015656 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4066c5a-25b5-4a23-80f7-ef95d3516d70-kubelet-config\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015666 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 12:11:08.016896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015675 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-conf-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015695 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-kubelet\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015713 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-slash\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015729 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-cnibin\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015738 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015750 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-netns\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015759 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-kubelet\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015776 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-cni-bin\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015775 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-slash\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015817 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-netns\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015811 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-conf-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015831 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-cni-bin\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015844 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-daemon-config\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015856 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-os-release\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015882 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-cni-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.017547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015899 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-cnibin\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015925 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502316a9-ade5-4bf4-b1f8-f6868b87987b-tmp-dir\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015907 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015937 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015962 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-modprobe-d\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015981 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-os-release\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.015992 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016022 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016066 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-multus-cni-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016054 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-device-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016111 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-cni-bin\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016126 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016143 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-device-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016171 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-var-lib-cni-bin\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b8bfc2d-09ba-415b-82f2-bc35e42c93ff-agent-certs\") pod \"konnectivity-agent-m7bb6\" (UID: \"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff\") " pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016225 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-system-cni-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016239 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016250 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016280 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-host\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016299 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-system-cni-dir\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016309 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-serviceca\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016323 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016338 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysconfig\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016356 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-run\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016373 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4066c5a-25b5-4a23-80f7-ef95d3516d70-dbus\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016400 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5db67\" (UniqueName: \"kubernetes.io/projected/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-kube-api-access-5db67\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016425 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysctl-d\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016453 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4066c5a-25b5-4a23-80f7-ef95d3516d70-dbus\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:08.018401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016460 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-sys\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016551 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016554 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016589 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-var-lib-kubelet\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016612 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-tuned\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016627 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wzjt\" (UniqueName: \"kubernetes.io/projected/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-kube-api-access-7wzjt\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016643 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-kubernetes\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016666 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-systemd\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016681 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-lib-modules\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016716 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-host\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016750 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-socket-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016776 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-k8s-cni-cncf-io\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016808 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c40b72bd-871f-46a8-b390-2afe6322872f-iptables-alerter-script\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016839 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-socket-dir\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.018868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016854 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f84c1aa-7a4b-438c-84fb-47deb0a69877-host-run-k8s-cni-cncf-io\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016895 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016922 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-etc-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016947 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cnibin\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016973 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhng6\" (UniqueName: \"kubernetes.io/projected/d58a9a1b-1177-4004-a9c0-c423078dace2-kube-api-access-rhng6\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016979 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-etc-openvswitch\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.016985 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.016998 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b8bfc2d-09ba-415b-82f2-bc35e42c93ff-konnectivity-ca\") pod \"konnectivity-agent-m7bb6\" (UID: \"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff\") " pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017010 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b0a1c8-c980-4d55-b819-a391a6a27fb7-cnibin\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017023 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcfx8\" (UniqueName: \"kubernetes.io/projected/5f84c1aa-7a4b-438c-84fb-47deb0a69877-kube-api-access-tcfx8\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.017045 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:08.517026537 +0000 UTC m=+2.010855527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017078 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppswd\" (UniqueName: \"kubernetes.io/projected/c40b72bd-871f-46a8-b390-2afe6322872f-kube-api-access-ppswd\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017104 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-sys-fs\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017183 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d58a9a1b-1177-4004-a9c0-c423078dace2-sys-fs\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.019264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017322 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c40b72bd-871f-46a8-b390-2afe6322872f-iptables-alerter-script\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.019735 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.017452 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b8bfc2d-09ba-415b-82f2-bc35e42c93ff-konnectivity-ca\") pod \"konnectivity-agent-m7bb6\" (UID: \"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff\") " pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:08.019735 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.019246 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-ovn-node-metrics-cert\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.019735 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.019332 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b8bfc2d-09ba-415b-82f2-bc35e42c93ff-agent-certs\") pod \"konnectivity-agent-m7bb6\" (UID: \"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff\") " pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:08.024048 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.024032 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:08.024115 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.024055 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:08.024115 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.024068 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:08.024191 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.024120 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:08.524109098 +0000 UTC m=+2.017938098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:08.026415 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.026383 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfw4d\" (UniqueName: \"kubernetes.io/projected/502316a9-ade5-4bf4-b1f8-f6868b87987b-kube-api-access-vfw4d\") pod \"node-resolver-pb5qj\" (UID: \"502316a9-ade5-4bf4-b1f8-f6868b87987b\") " pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.027256 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.027229 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppswd\" (UniqueName: \"kubernetes.io/projected/c40b72bd-871f-46a8-b390-2afe6322872f-kube-api-access-ppswd\") pod \"iptables-alerter-ks9gx\" (UID: \"c40b72bd-871f-46a8-b390-2afe6322872f\") " pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.028252 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.028227 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdddn\" (UniqueName: \"kubernetes.io/projected/a385e43d-14a9-4f43-a962-a08a8523cb5e-kube-api-access-bdddn\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:08.028753 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.028725 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7lm2\" (UniqueName: \"kubernetes.io/projected/91b0a1c8-c980-4d55-b819-a391a6a27fb7-kube-api-access-v7lm2\") pod \"multus-additional-cni-plugins-7k4bk\" (UID: \"91b0a1c8-c980-4d55-b819-a391a6a27fb7\") " pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.029190 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.029155 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhng6\" (UniqueName: \"kubernetes.io/projected/d58a9a1b-1177-4004-a9c0-c423078dace2-kube-api-access-rhng6\") pod \"aws-ebs-csi-driver-node-2whtg\" (UID: \"d58a9a1b-1177-4004-a9c0-c423078dace2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.031343 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.031315 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcfx8\" (UniqueName: \"kubernetes.io/projected/5f84c1aa-7a4b-438c-84fb-47deb0a69877-kube-api-access-tcfx8\") pod \"multus-bsc4n\" (UID: \"5f84c1aa-7a4b-438c-84fb-47deb0a69877\") " pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.032330 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.032311 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5db67\" (UniqueName: \"kubernetes.io/projected/1c0acb4d-b957-4a5a-bc36-a077512a9cb3-kube-api-access-5db67\") pod \"ovnkube-node-xwhch\" (UID: \"1c0acb4d-b957-4a5a-bc36-a077512a9cb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.118565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118457 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-modprobe-d\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118494 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-host\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.118565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118510 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-serviceca\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.118565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysconfig\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118557 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-run\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118589 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-host\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118621 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-run\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118623 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysconfig\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118625 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-modprobe-d\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118651 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysctl-d\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-sys\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118695 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-var-lib-kubelet\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-tuned\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118755 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-sys\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118775 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-var-lib-kubelet\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118812 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysctl-d\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118816 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wzjt\" (UniqueName: \"kubernetes.io/projected/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-kube-api-access-7wzjt\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118857 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-kubernetes\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118886 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-systemd\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.118934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118911 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-lib-modules\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118940 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-host\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118966 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-kubernetes\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.118994 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-systemd\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119044 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttq7j\" (UniqueName: \"kubernetes.io/projected/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-kube-api-access-ttq7j\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119048 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-serviceca\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119075 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysctl-conf\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119077 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-lib-modules\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119100 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-tmp\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119109 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-host\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.119656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.119190 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-sysctl-conf\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.120939 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.120922 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-etc-tuned\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.121121 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.121101 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-tmp\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.128590 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.128570 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wzjt\" (UniqueName: \"kubernetes.io/projected/72d97c2b-a9e5-4932-a0c0-862f7ed077fa-kube-api-access-7wzjt\") pod \"tuned-55hxx\" (UID: \"72d97c2b-a9e5-4932-a0c0-862f7ed077fa\") " pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.128697 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.128682 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttq7j\" (UniqueName: \"kubernetes.io/projected/a5f7f769-70f8-4cfe-a810-ff6d3e476ce4-kube-api-access-ttq7j\") pod \"node-ca-wxzfm\" (UID: \"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4\") " pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.203776 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.203744 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:08.213820 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.213801 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:08.229382 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.229356 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ks9gx" Jul 21 12:11:08.243686 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.243656 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pb5qj" Jul 21 12:11:08.260879 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.260851 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bsc4n" Jul 21 12:11:08.277206 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.277177 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" Jul 21 12:11:08.289603 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.289579 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" Jul 21 12:11:08.305455 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.305417 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wxzfm" Jul 21 12:11:08.309300 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.309283 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-55hxx" Jul 21 12:11:08.521793 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.521754 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:08.521793 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.521806 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:08.522022 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.521920 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:08.522022 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.521990 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:09.52197388 +0000 UTC m=+3.015802853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:08.522022 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.521919 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:08.522181 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.522080 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:09.522063263 +0000 UTC m=+3.015892241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:08.622457 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.622418 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:08.622600 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.622566 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:08.622600 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.622583 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:08.622600 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.622594 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:08.622728 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:08.622645 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:09.622631206 +0000 UTC m=+3.116460182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:08.967142 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:08.967109 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a6cf30d9fe955de2f6516c2a1ccf33.slice/crio-cf4cc6d911d099dc06f64093cc3b7ff84a283da69725368f5cfcc5ef259ddc4f WatchSource:0}: Error finding container cf4cc6d911d099dc06f64093cc3b7ff84a283da69725368f5cfcc5ef259ddc4f: Status 404 returned error can't find the container with id cf4cc6d911d099dc06f64093cc3b7ff84a283da69725368f5cfcc5ef259ddc4f Jul 21 12:11:08.967783 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:08.967618 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40b72bd_871f_46a8_b390_2afe6322872f.slice/crio-4872ea68e611a04543155a35d10fc01411d9e01d4aea0af5f9736575a6fb03a4 WatchSource:0}: Error finding container 4872ea68e611a04543155a35d10fc01411d9e01d4aea0af5f9736575a6fb03a4: Status 404 returned error can't find the container with id 4872ea68e611a04543155a35d10fc01411d9e01d4aea0af5f9736575a6fb03a4 Jul 21 12:11:08.968776 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:08.968756 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c0acb4d_b957_4a5a_bc36_a077512a9cb3.slice/crio-67e9ff98ec08f76e1002d34b71c90959cf0144ace57d6604091e8c937da70f3f WatchSource:0}: Error finding container 67e9ff98ec08f76e1002d34b71c90959cf0144ace57d6604091e8c937da70f3f: Status 404 returned error can't find the container with id 67e9ff98ec08f76e1002d34b71c90959cf0144ace57d6604091e8c937da70f3f Jul 21 12:11:08.980123 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.980087 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:11:08.987030 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:08.987007 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b0a1c8_c980_4d55_b819_a391a6a27fb7.slice/crio-87539a4ca739a2d311df93e9b92159cd6a1c897579eb268f4ca1a4d95f14ca71 WatchSource:0}: Error finding container 87539a4ca739a2d311df93e9b92159cd6a1c897579eb268f4ca1a4d95f14ca71: Status 404 returned error can't find the container with id 87539a4ca739a2d311df93e9b92159cd6a1c897579eb268f4ca1a4d95f14ca71 Jul 21 12:11:08.990559 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.990514 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:06:08 +0000 UTC" deadline="2027-12-16 03:07:34.370927494 +0000 UTC" Jul 21 12:11:08.990559 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:08.990555 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12302h56m25.380374425s" Jul 21 12:11:09.041922 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:09.041866 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58a9a1b_1177_4004_a9c0_c423078dace2.slice/crio-8db0cf4d30ce01174a784ebbccb5ce44fa46306342fc4e35cbbc4deb6c9b7cfa WatchSource:0}: Error finding container 8db0cf4d30ce01174a784ebbccb5ce44fa46306342fc4e35cbbc4deb6c9b7cfa: Status 404 returned error can't find the container with id 8db0cf4d30ce01174a784ebbccb5ce44fa46306342fc4e35cbbc4deb6c9b7cfa Jul 21 12:11:09.043089 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:09.043058 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc81c72d91b2f4557f0246b9d50dc4e.slice/crio-f9d1ebb627e9926a31725612696f201af70bcf46cd56e4cbdffdcd4687519072 WatchSource:0}: Error finding container f9d1ebb627e9926a31725612696f201af70bcf46cd56e4cbdffdcd4687519072: Status 404 returned error can't find the container with id f9d1ebb627e9926a31725612696f201af70bcf46cd56e4cbdffdcd4687519072 Jul 21 12:11:09.043815 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:09.043794 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502316a9_ade5_4bf4_b1f8_f6868b87987b.slice/crio-79795044d9ee0a56ba67541b8e66ee8f89d5da6f9a3471f269f6e388ee6c896c WatchSource:0}: Error finding container 79795044d9ee0a56ba67541b8e66ee8f89d5da6f9a3471f269f6e388ee6c896c: Status 404 returned error can't find the container with id 79795044d9ee0a56ba67541b8e66ee8f89d5da6f9a3471f269f6e388ee6c896c Jul 21 12:11:09.113929 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.113888 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:09.114112 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.114004 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:09.118030 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.117979 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"67e9ff98ec08f76e1002d34b71c90959cf0144ace57d6604091e8c937da70f3f"} Jul 21 12:11:09.119017 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.118994 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" event={"ID":"e0a6cf30d9fe955de2f6516c2a1ccf33","Type":"ContainerStarted","Data":"cf4cc6d911d099dc06f64093cc3b7ff84a283da69725368f5cfcc5ef259ddc4f"} Jul 21 12:11:09.120010 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.119981 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" event={"ID":"8dc81c72d91b2f4557f0246b9d50dc4e","Type":"ContainerStarted","Data":"f9d1ebb627e9926a31725612696f201af70bcf46cd56e4cbdffdcd4687519072"} Jul 21 12:11:09.120931 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.120910 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pb5qj" event={"ID":"502316a9-ade5-4bf4-b1f8-f6868b87987b","Type":"ContainerStarted","Data":"79795044d9ee0a56ba67541b8e66ee8f89d5da6f9a3471f269f6e388ee6c896c"} Jul 21 12:11:09.122065 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.122042 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" event={"ID":"d58a9a1b-1177-4004-a9c0-c423078dace2","Type":"ContainerStarted","Data":"8db0cf4d30ce01174a784ebbccb5ce44fa46306342fc4e35cbbc4deb6c9b7cfa"} Jul 21 12:11:09.123002 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.122980 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerStarted","Data":"87539a4ca739a2d311df93e9b92159cd6a1c897579eb268f4ca1a4d95f14ca71"} Jul 21 12:11:09.123931 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.123906 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ks9gx" event={"ID":"c40b72bd-871f-46a8-b390-2afe6322872f","Type":"ContainerStarted","Data":"4872ea68e611a04543155a35d10fc01411d9e01d4aea0af5f9736575a6fb03a4"} Jul 21 12:11:09.136399 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:09.136369 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f7f769_70f8_4cfe_a810_ff6d3e476ce4.slice/crio-15bd828d126aa8764dcf368025c7644dd6c83599e8fefc8760c3958e8c301202 WatchSource:0}: Error finding container 15bd828d126aa8764dcf368025c7644dd6c83599e8fefc8760c3958e8c301202: Status 404 returned error can't find the container with id 15bd828d126aa8764dcf368025c7644dd6c83599e8fefc8760c3958e8c301202 Jul 21 12:11:09.188126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.188093 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:09.235535 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:09.235504 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f84c1aa_7a4b_438c_84fb_47deb0a69877.slice/crio-fc1cc637ad98d7dac16f33a75de0f61fe4a22a71c9762914c0bccef37ff4b7fd WatchSource:0}: Error finding container fc1cc637ad98d7dac16f33a75de0f61fe4a22a71c9762914c0bccef37ff4b7fd: Status 404 returned error can't find the container with id fc1cc637ad98d7dac16f33a75de0f61fe4a22a71c9762914c0bccef37ff4b7fd Jul 21 12:11:09.336386 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:09.336361 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d97c2b_a9e5_4932_a0c0_862f7ed077fa.slice/crio-2c32b65fdcf3c760ccb45ae0905a87dce11bc7b1f2993a7219b98a9f2fc3127d WatchSource:0}: Error finding container 2c32b65fdcf3c760ccb45ae0905a87dce11bc7b1f2993a7219b98a9f2fc3127d: Status 404 returned error can't find the container with id 2c32b65fdcf3c760ccb45ae0905a87dce11bc7b1f2993a7219b98a9f2fc3127d Jul 21 12:11:09.531770 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.531673 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:09.531770 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.531723 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:09.531941 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.531833 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:09.531941 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.531898 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:11.53188172 +0000 UTC m=+5.025710698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:09.531941 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.531838 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:09.532055 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.531973 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:11.53195937 +0000 UTC m=+5.025788345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:09.632826 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.632786 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:09.633089 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.633001 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:09.633089 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.633031 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:09.633089 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.633045 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:09.633269 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:09.633120 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:11.63309838 +0000 UTC m=+5.126927356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:09.991214 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.991148 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:06:08 +0000 UTC" deadline="2027-12-24 19:28:46.903913702 +0000 UTC" Jul 21 12:11:09.991214 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:09.991193 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12511h17m36.91272576s" Jul 21 12:11:10.093305 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.093270 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:10.114092 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.114057 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:10.114278 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:10.114221 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:10.114754 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.114728 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:10.114889 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:10.114862 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:10.132719 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.132686 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-55hxx" event={"ID":"72d97c2b-a9e5-4932-a0c0-862f7ed077fa","Type":"ContainerStarted","Data":"2c32b65fdcf3c760ccb45ae0905a87dce11bc7b1f2993a7219b98a9f2fc3127d"} Jul 21 12:11:10.137835 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.137807 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bsc4n" event={"ID":"5f84c1aa-7a4b-438c-84fb-47deb0a69877","Type":"ContainerStarted","Data":"fc1cc637ad98d7dac16f33a75de0f61fe4a22a71c9762914c0bccef37ff4b7fd"} Jul 21 12:11:10.143618 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.143588 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wxzfm" event={"ID":"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4","Type":"ContainerStarted","Data":"15bd828d126aa8764dcf368025c7644dd6c83599e8fefc8760c3958e8c301202"} Jul 21 12:11:10.160673 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.160596 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m7bb6" event={"ID":"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff","Type":"ContainerStarted","Data":"fd903fae7a4215cbdfb3a3fdeb95fe9e1609c2afd7d92ff956e0157582e719cb"} Jul 21 12:11:10.249455 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:10.249372 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:11:11.114183 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:11.114152 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:11.114756 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.114265 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:11.548738 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:11.548690 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:11.548944 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:11.548760 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:11.548944 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.548904 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:11.549066 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.548969 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:15.548949296 +0000 UTC m=+9.042778271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:11.549473 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.549426 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:11.549625 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.549484 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:15.549468182 +0000 UTC m=+9.043297156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:11.649990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:11.649948 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:11.650185 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.650146 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:11.650185 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.650170 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:11.650326 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.650183 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:11.650326 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:11.650258 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:15.650237519 +0000 UTC m=+9.144066503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:12.114089 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:12.113965 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:12.114276 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:12.114123 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:12.114990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:12.114552 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:12.114990 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:12.114651 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:13.113596 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:13.113559 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:13.113791 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:13.113719 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:14.114779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:14.114053 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:14.114779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:14.114197 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:14.114779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:14.114053 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:14.114779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:14.114730 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:15.114015 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:15.113984 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:15.114216 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.114114 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:15.584191 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:15.584154 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:15.584692 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:15.584212 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:15.584692 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.584329 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:15.584692 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.584381 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:15.584692 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.584410 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:23.584387665 +0000 UTC m=+17.078216642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:15.584692 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.584450 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:23.584416797 +0000 UTC m=+17.078245794 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:15.685693 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:15.685080 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:15.685693 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.685261 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:15.685693 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.685282 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:15.685693 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.685292 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:15.685693 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:15.685339 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:23.685322886 +0000 UTC m=+17.179151858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:16.114125 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:16.114090 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:16.114316 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:16.114259 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:16.114379 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:16.114322 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:16.114509 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:16.114481 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:17.115004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:17.114769 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:17.115004 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:17.114892 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:18.113633 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:18.113599 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:18.113826 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:18.113729 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:18.113826 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:18.113603 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:18.113941 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:18.113894 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:19.113695 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:19.113609 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:19.114134 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:19.113762 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:20.114167 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:20.113854 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:20.114167 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:20.114020 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:20.114167 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:20.113863 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:20.114167 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:20.114133 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:21.113920 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:21.113882 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:21.114089 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:21.113993 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:22.114305 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:22.114265 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:22.114750 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:22.114395 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:22.114750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:22.114459 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:22.114750 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:22.114574 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:23.114064 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:23.113995 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:23.114256 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.114162 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:23.645953 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:23.645920 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:23.645953 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:23.645962 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:23.646499 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.646084 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:23.646499 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.646086 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:23.646499 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.646143 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:39.646128258 +0000 UTC m=+33.139957228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:23.646499 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.646157 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:39.646150643 +0000 UTC m=+33.139979613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:23.747122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:23.747087 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:23.747294 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.747221 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:23.747294 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.747235 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:23.747294 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.747243 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:23.747418 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:23.747301 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:39.747286851 +0000 UTC m=+33.241115825 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:24.114215 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:24.114177 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:24.114393 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:24.114296 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:24.114393 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:24.114353 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:24.114504 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:24.114473 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:25.114074 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:25.114038 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:25.114443 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:25.114141 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:26.114327 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:26.114114 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:26.114768 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:26.114187 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:26.114768 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:26.114454 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:26.114768 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:26.114518 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:27.114800 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:27.114770 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:27.115277 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:27.114873 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:28.114048 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:28.114015 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:28.114228 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:28.114021 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:28.114228 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:28.114140 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:28.114326 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:28.114234 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:29.113549 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:29.113513 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:29.113991 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:29.113645 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:30.113530 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.113506 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:30.113691 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.113504 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:30.113691 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:30.113614 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:30.114221 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:30.113718 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:30.208105 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.208075 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"fa5975ce87ecbc754ea2a021476e23b5e73c333278d699068d939cfa6f78db0b"} Jul 21 12:11:30.208211 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.208155 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"6105395175de51b76e47d92a2e5f9c994f51bc7bbfb21f89d5668cc8e792b598"} Jul 21 12:11:30.209699 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.209495 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-55hxx" event={"ID":"72d97c2b-a9e5-4932-a0c0-862f7ed077fa","Type":"ContainerStarted","Data":"cc5008d2676ae738fa78ba9ff41fc85299a6fe50e607c2879f336e1ec57259b4"} Jul 21 12:11:30.211935 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.211916 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bsc4n" event={"ID":"5f84c1aa-7a4b-438c-84fb-47deb0a69877","Type":"ContainerStarted","Data":"3b73f8fe83fe98c49ef44a2729ac4c0c7fcc6b3ae30be4ffa510ef4ed6152418"} Jul 21 12:11:30.213214 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.213193 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" event={"ID":"8dc81c72d91b2f4557f0246b9d50dc4e","Type":"ContainerStarted","Data":"a8f877053e36e976e30a1e98d3ca493bca01d7b0b6152ace81748ecce6c6cc61"} Jul 21 12:11:30.239647 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.239602 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-55hxx" podStartSLOduration=2.074062466 podStartE2EDuration="22.239588187s" totalImagesPullingTime="20.165525721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:30.23939868 +0000 UTC m=+23.733227668" watchObservedRunningTime="2026-07-21 12:11:30.239588187 +0000 UTC m=+23.733417180" Jul 21 12:11:30.255530 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:30.255479 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-33.ec2.internal" podStartSLOduration=22.255462393 podStartE2EDuration="22.255462393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:30.255352109 +0000 UTC m=+23.749181102" watchObservedRunningTime="2026-07-21 12:11:30.255462393 +0000 UTC m=+23.749291386" Jul 21 12:11:31.114527 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.114271 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:31.115141 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:31.114562 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:31.216674 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.216639 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pb5qj" event={"ID":"502316a9-ade5-4bf4-b1f8-f6868b87987b","Type":"ContainerStarted","Data":"db9027574c015ff88f6c9afe2e0ba5456cd39d43dca06d155abd3959544bc780"} Jul 21 12:11:31.218058 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.218029 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ks9gx" event={"ID":"c40b72bd-871f-46a8-b390-2afe6322872f","Type":"ContainerStarted","Data":"55e350369cf9fdc3c72edd16b1c12137e6b26867a23ac350a20cdf763b21856d"} Jul 21 12:11:31.219311 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.219289 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m7bb6" event={"ID":"4b8bfc2d-09ba-415b-82f2-bc35e42c93ff","Type":"ContainerStarted","Data":"e02fe5e0f1be33f44058f195ab9c867284c2c147e09114fc5bde42cb4620fcd0"} Jul 21 12:11:31.220757 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.220737 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" event={"ID":"d58a9a1b-1177-4004-a9c0-c423078dace2","Type":"ContainerStarted","Data":"89bf2cefdd1d6db81a6c0b9d42acb3ef05aa8f508598211672bb1e7f0511ab81"} Jul 21 12:11:31.222121 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.222099 2550 generic.go:356] "Generic (PLEG): container finished" podID="91b0a1c8-c980-4d55-b819-a391a6a27fb7" containerID="1c72946fa16b2c27e6be8fb1c555b39fd16ceb96608d28461d3106c50d8c07fe" exitCode=0 Jul 21 12:11:31.222186 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.222175 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerDied","Data":"1c72946fa16b2c27e6be8fb1c555b39fd16ceb96608d28461d3106c50d8c07fe"} Jul 21 12:11:31.225175 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225147 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:11:31.225485 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225460 2550 generic.go:356] "Generic (PLEG): container finished" podID="1c0acb4d-b957-4a5a-bc36-a077512a9cb3" containerID="fa5975ce87ecbc754ea2a021476e23b5e73c333278d699068d939cfa6f78db0b" exitCode=1 Jul 21 12:11:31.225571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225527 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerDied","Data":"fa5975ce87ecbc754ea2a021476e23b5e73c333278d699068d939cfa6f78db0b"} Jul 21 12:11:31.225615 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225594 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"8662095652ec1cc6786bae8b829d70f122ed67104551099d447197c7ecd1b5f5"} Jul 21 12:11:31.225657 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225637 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"db9f34888b85290b457453f7110f4df9c08c8c22587e181cfcf056994128ecb8"} Jul 21 12:11:31.225690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225666 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"77444577dde50964dcaa7c05a0a3309fe17863a7bd62307ba0d164a3c9f07d34"} Jul 21 12:11:31.225727 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.225700 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"02dbb3660e51cbcfd7b4637a53d99c7cc6cbcd193297e8175cf040b6454e5a8a"} Jul 21 12:11:31.226738 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.226718 2550 generic.go:356] "Generic (PLEG): container finished" podID="e0a6cf30d9fe955de2f6516c2a1ccf33" containerID="1a52cd2d0c98395e9dc6ee21e8c28ec89d43ef21756128e142991f3365da2aee" exitCode=0 Jul 21 12:11:31.226827 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.226788 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" event={"ID":"e0a6cf30d9fe955de2f6516c2a1ccf33","Type":"ContainerDied","Data":"1a52cd2d0c98395e9dc6ee21e8c28ec89d43ef21756128e142991f3365da2aee"} Jul 21 12:11:31.228151 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.228121 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wxzfm" event={"ID":"a5f7f769-70f8-4cfe-a810-ff6d3e476ce4","Type":"ContainerStarted","Data":"d96ed5eca039b4ab2e5ca8b88e28723004441b957ea593e6dd8a76f1b3e0ae9c"} Jul 21 12:11:31.233655 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.233622 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bsc4n" podStartSLOduration=3.9518922979999997 podStartE2EDuration="24.233609239s" totalImagesPullingTime="20.281716941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:30.283857718 +0000 UTC m=+23.777686712" watchObservedRunningTime="2026-07-21 12:11:31.233609239 +0000 UTC m=+24.727438231" Jul 21 12:11:31.234057 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.234036 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pb5qj" podStartSLOduration=3.775594886 podStartE2EDuration="24.234028411s" totalImagesPullingTime="20.458433525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:31.233527674 +0000 UTC m=+24.727356668" watchObservedRunningTime="2026-07-21 12:11:31.234028411 +0000 UTC m=+24.727857404" Jul 21 12:11:31.249308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.249275 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ks9gx" podStartSLOduration=3.745218967 podStartE2EDuration="24.249264311s" totalImagesPullingTime="20.504045344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:31.248947013 +0000 UTC m=+24.742776007" watchObservedRunningTime="2026-07-21 12:11:31.249264311 +0000 UTC m=+24.743093303" Jul 21 12:11:31.301909 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.301683 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m7bb6" podStartSLOduration=4.271870465 podStartE2EDuration="24.301667836s" totalImagesPullingTime="20.029797371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:31.301540852 +0000 UTC m=+24.795369844" watchObservedRunningTime="2026-07-21 12:11:31.301667836 +0000 UTC m=+24.795496829" Jul 21 12:11:31.316659 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.316621 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wxzfm" podStartSLOduration=2.970297941 podStartE2EDuration="23.316606556s" totalImagesPullingTime="20.346308615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:31.316416608 +0000 UTC m=+24.810245602" watchObservedRunningTime="2026-07-21 12:11:31.316606556 +0000 UTC m=+24.810435549" Jul 21 12:11:31.574297 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.574259 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:31.574914 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:31.574890 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:32.064945 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.064918 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 12:11:32.114470 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.114439 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:32.114659 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.114437 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:32.114659 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:32.114564 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:32.115011 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:32.114669 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:32.231809 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.231720 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" event={"ID":"d58a9a1b-1177-4004-a9c0-c423078dace2","Type":"ContainerStarted","Data":"0823d79015b2553d7b350fabfe3a8e6a3a1df9feaeb581449c1937f3aab8ff69"} Jul 21 12:11:32.233584 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.233555 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" event={"ID":"e0a6cf30d9fe955de2f6516c2a1ccf33","Type":"ContainerStarted","Data":"34e491b8465166219ab60410a952e4b6faf7a9f9b875b529443518b4129b3246"} Jul 21 12:11:32.234159 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.234131 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:32.234333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.234317 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m7bb6" Jul 21 12:11:32.249113 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:32.249055 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-33.ec2.internal" podStartSLOduration=24.248996232 podStartE2EDuration="24.248996232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:32.24880467 +0000 UTC m=+25.742633663" watchObservedRunningTime="2026-07-21 12:11:32.248996232 +0000 UTC m=+25.742825225" Jul 21 12:11:33.015338 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:33.015220 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:11:32.064939517Z","UUID":"ce666c3e-052a-48b9-adc4-14d14748498f","Handler":null,"Name":"","Endpoint":""} Jul 21 12:11:33.017223 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:33.017143 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 12:11:33.017223 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:33.017188 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 12:11:33.113652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:33.113585 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:33.113795 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:33.113699 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:33.239097 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:33.239066 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:11:33.239600 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:33.239528 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"63754606647f5caccc680e660f3b0d01ff3004baed6fdaab575be1760ef799ed"} Jul 21 12:11:34.114364 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:34.114324 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:34.114556 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:34.114337 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:34.114556 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:34.114444 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:34.114646 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:34.114548 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:34.244158 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:34.244125 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" event={"ID":"d58a9a1b-1177-4004-a9c0-c423078dace2","Type":"ContainerStarted","Data":"11071578a6952b62b4c48e7992ef6e589bde87ce4fb46038f4a1d08a1fce106c"} Jul 21 12:11:34.264767 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:34.264709 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2whtg" podStartSLOduration=3.222648867 podStartE2EDuration="26.264672413s" totalImagesPullingTime="23.042023546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:34.264167291 +0000 UTC m=+27.757996287" watchObservedRunningTime="2026-07-21 12:11:34.264672413 +0000 UTC m=+27.758501449" Jul 21 12:11:35.114204 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:35.114165 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:35.114380 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:35.114292 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:36.114563 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:36.114379 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:36.115004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:36.114379 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:36.115004 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:36.114647 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:36.115004 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:36.114728 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:37.114123 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.114084 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:37.114363 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:37.114172 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:37.252114 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.252078 2550 generic.go:356] "Generic (PLEG): container finished" podID="91b0a1c8-c980-4d55-b819-a391a6a27fb7" containerID="998ce31784a8172358d829912ad2a2b44c806954c909f0b3854e6fd46df3e9c4" exitCode=0 Jul 21 12:11:37.252877 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.252165 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerDied","Data":"998ce31784a8172358d829912ad2a2b44c806954c909f0b3854e6fd46df3e9c4"} Jul 21 12:11:37.255578 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.255560 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:11:37.255996 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.255974 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"b88316baf539bf5fe9748c6a45872a51f4c7d614dee002362a64ab5a622a2c9c"} Jul 21 12:11:37.256245 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.256223 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:37.256354 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.256294 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:37.256354 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.256333 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:37.256451 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.256443 2550 scope.go:117] "RemoveContainer" containerID="fa5975ce87ecbc754ea2a021476e23b5e73c333278d699068d939cfa6f78db0b" Jul 21 12:11:37.274217 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.274193 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:37.274377 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:37.274360 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:11:38.114088 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.113911 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:38.114311 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.113926 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:38.114311 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:38.114190 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:38.114311 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:38.114223 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:38.260986 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.260962 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:11:38.261366 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.261290 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" event={"ID":"1c0acb4d-b957-4a5a-bc36-a077512a9cb3","Type":"ContainerStarted","Data":"36c5a5b1154eb680bdb66ef7790e7c75137e474c71d60222ed5170cc660e42bb"} Jul 21 12:11:38.290564 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.289342 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" podStartSLOduration=10.333136177 podStartE2EDuration="31.289319026s" totalImagesPullingTime="20.956182849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:38.288680785 +0000 UTC m=+31.782509803" watchObservedRunningTime="2026-07-21 12:11:38.289319026 +0000 UTC m=+31.783148020" Jul 21 12:11:38.628962 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.628934 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rrb6l"] Jul 21 12:11:38.629130 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.629070 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:38.629177 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:38.629147 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:38.631515 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.631491 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bgmwr"] Jul 21 12:11:38.631623 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.631607 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:38.631736 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:38.631711 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:38.634325 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.634303 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jknc7"] Jul 21 12:11:38.634437 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:38.634377 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:38.634508 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:38.634460 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:39.266001 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:39.265959 2550 generic.go:356] "Generic (PLEG): container finished" podID="91b0a1c8-c980-4d55-b819-a391a6a27fb7" containerID="9b198b3490c1986fc9da3d6533ee05181e25c85fd97acf75e22a6d9dc57d8f36" exitCode=0 Jul 21 12:11:39.266471 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:39.266035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerDied","Data":"9b198b3490c1986fc9da3d6533ee05181e25c85fd97acf75e22a6d9dc57d8f36"} Jul 21 12:11:39.667320 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:39.667260 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:39.667320 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:39.667313 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:39.667605 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.667411 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:39.667605 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.667459 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:39.667605 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.667527 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs podName:a385e43d-14a9-4f43-a962-a08a8523cb5e nodeName:}" failed. No retries permitted until 2026-07-21 12:12:11.667505916 +0000 UTC m=+65.161334900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs") pod "network-metrics-daemon-bgmwr" (UID: "a385e43d-14a9-4f43-a962-a08a8523cb5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:11:39.667605 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.667545 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret podName:a4066c5a-25b5-4a23-80f7-ef95d3516d70 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:11.667536812 +0000 UTC m=+65.161365785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret") pod "global-pull-secret-syncer-rrb6l" (UID: "a4066c5a-25b5-4a23-80f7-ef95d3516d70") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:11:39.768714 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:39.768680 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:39.768927 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.768832 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:11:39.768927 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.768856 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:11:39.768927 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.768867 2550 projected.go:196] Error preparing data for projected volume kube-api-access-sjjrl for pod openshift-network-diagnostics/network-check-target-jknc7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:39.768927 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:39.768927 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl podName:8888d463-620e-41f9-8f23-222532c774c0 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:11.768911225 +0000 UTC m=+65.262740197 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sjjrl" (UniqueName: "kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl") pod "network-check-target-jknc7" (UID: "8888d463-620e-41f9-8f23-222532c774c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:11:40.113958 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:40.113920 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:40.114122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:40.113920 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:40.114122 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:40.114041 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:40.114213 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:40.114134 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:40.114213 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:40.113920 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:40.114277 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:40.114246 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:41.272767 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:41.272729 2550 generic.go:356] "Generic (PLEG): container finished" podID="91b0a1c8-c980-4d55-b819-a391a6a27fb7" containerID="21c65b229e1989e592d03ed65e09e4e9757d6c58766b766cdc9839da7cdb083f" exitCode=0 Jul 21 12:11:41.273228 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:41.272794 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerDied","Data":"21c65b229e1989e592d03ed65e09e4e9757d6c58766b766cdc9839da7cdb083f"} Jul 21 12:11:42.113740 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.113697 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:42.113740 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.113741 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:42.114003 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.113746 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:42.114003 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.113816 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rrb6l" podUID="a4066c5a-25b5-4a23-80f7-ef95d3516d70" Jul 21 12:11:42.114003 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.113933 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bgmwr" podUID="a385e43d-14a9-4f43-a962-a08a8523cb5e" Jul 21 12:11:42.114127 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.114026 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jknc7" podUID="8888d463-620e-41f9-8f23-222532c774c0" Jul 21 12:11:42.760587 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.760558 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-33.ec2.internal" event="NodeReady" Jul 21 12:11:42.761023 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.760708 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 12:11:42.795550 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.795519 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp"] Jul 21 12:11:42.798257 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.798237 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.800730 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.800703 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2"] Jul 21 12:11:42.801073 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.801047 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8qzwk\"" Jul 21 12:11:42.801229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.801072 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 21 12:11:42.801229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.801072 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.801229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.801131 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.801229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.801054 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 21 12:11:42.803781 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.803748 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z"] Jul 21 12:11:42.803890 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.803834 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:42.808249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.807354 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.808249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.807773 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 21 12:11:42.808249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.808105 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qpjlh\"" Jul 21 12:11:42.808403 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.808347 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.810000 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.809980 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c848ff744-k8fkq"] Jul 21 12:11:42.810103 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.810081 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" Jul 21 12:11:42.812470 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.812453 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.812576 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.812529 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hq68p\"" Jul 21 12:11:42.812953 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.812932 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.813475 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.813419 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb"] Jul 21 12:11:42.813568 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.813515 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.816171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.816148 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 12:11:42.816882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.816716 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l"] Jul 21 12:11:42.817074 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.817056 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 12:11:42.817268 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.817254 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 12:11:42.817392 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.817376 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l94b2\"" Jul 21 12:11:42.818044 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.817610 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.820057 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.820036 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9f8jw\"" Jul 21 12:11:42.820179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.820061 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-zsqww"] Jul 21 12:11:42.820179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.820081 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 21 12:11:42.820179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.820124 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 21 12:11:42.820409 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.820388 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:42.822542 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.822454 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.822722 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.822702 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.824347 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.824328 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-568fbddcbd-x8pzc"] Jul 21 12:11:42.824547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.824529 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.826924 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.826899 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 12:11:42.827130 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.827115 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 21 12:11:42.827253 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.827232 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.827358 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.827315 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.827562 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.827543 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 12:11:42.827645 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.827599 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 12:11:42.827990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.827972 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r9cg8\"" Jul 21 12:11:42.829608 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.829567 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-dvskp"] Jul 21 12:11:42.829719 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.829680 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:42.830862 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.830841 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.830967 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.830842 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kd47m\"" Jul 21 12:11:42.831102 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.831039 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.831500 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.831419 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 21 12:11:42.833328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833309 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp"] Jul 21 12:11:42.833453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833343 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2"] Jul 21 12:11:42.833453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833375 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z"] Jul 21 12:11:42.833453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833393 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb"] Jul 21 12:11:42.833453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833409 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c848ff744-k8fkq"] Jul 21 12:11:42.833453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833445 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-dvskp"] Jul 21 12:11:42.833698 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833464 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-568fbddcbd-x8pzc"] Jul 21 12:11:42.833698 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833485 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nrn6d"] Jul 21 12:11:42.833849 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.833829 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:42.835305 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.835262 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 21 12:11:42.835484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.835336 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.836396 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.835678 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 21 12:11:42.836396 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.836256 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.836563 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.836453 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 21 12:11:42.836944 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.836923 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4t9m4\"" Jul 21 12:11:42.837864 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.837844 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 21 12:11:42.840349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.839551 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.842525 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.841365 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z"] Jul 21 12:11:42.843319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.843141 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:42.843991 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.843972 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pm78n\"" Jul 21 12:11:42.844126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.844110 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 12:11:42.844292 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.844272 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.844597 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.844568 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 12:11:42.846120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.846097 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 12:11:42.846566 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.846540 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 12:11:42.847170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.846372 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:42.847785 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.846500 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fnnf7\"" Jul 21 12:11:42.847973 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.846287 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf"] Jul 21 12:11:42.848192 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.848164 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 21 12:11:42.851061 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.851030 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ccdv6\"" Jul 21 12:11:42.851379 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.851259 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 12:11:42.852059 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.851905 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 12:11:42.852355 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.852337 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 12:11:42.853879 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.853861 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fl9m4"] Jul 21 12:11:42.854000 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.853982 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" Jul 21 12:11:42.856550 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.856529 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.856808 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.856757 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qj7nj\"" Jul 21 12:11:42.857046 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.857026 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.857672 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.857653 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l"] Jul 21 12:11:42.858075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.857689 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z"] Jul 21 12:11:42.858075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.857710 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-zsqww"] Jul 21 12:11:42.858075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.857733 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nrn6d"] Jul 21 12:11:42.858075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.857814 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:42.861024 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.860966 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fl9m4"] Jul 21 12:11:42.862555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.861455 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 12:11:42.862555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.861677 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6rff6\"" Jul 21 12:11:42.862555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.861950 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 12:11:42.862555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.862275 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 12:11:42.862555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.862499 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf"] Jul 21 12:11:42.891094 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891069 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-config\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.891205 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891107 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxj4w\" (UniqueName: \"kubernetes.io/projected/30ad9c8e-526a-4cbd-973d-8719f34ead2e-kube-api-access-xxj4w\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:42.891205 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891185 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfr8g\" (UniqueName: \"kubernetes.io/projected/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-kube-api-access-dfr8g\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.891349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891231 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdfgl\" (UniqueName: \"kubernetes.io/projected/67b67449-f536-4272-bb03-f8224c8e196e-kube-api-access-gdfgl\") pod \"volume-data-source-validator-78cf4c6f6-8hg6z\" (UID: \"67b67449-f536-4272-bb03-f8224c8e196e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" Jul 21 12:11:42.891349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891306 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891334 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-ca-trust-extracted\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891574 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891364 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-serving-cert\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.891574 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891435 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-image-registry-private-configuration\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891574 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891462 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-certificates\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891574 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891510 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-trusted-ca\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891574 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891543 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-installation-pull-secrets\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891574 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891570 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-bound-sa-token\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891593 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khxh8\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-kube-api-access-khxh8\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.891853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891620 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7135034a-0b9e-44b6-9619-93b88cf3a501-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.891853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891647 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdm8b\" (UniqueName: \"kubernetes.io/projected/7135034a-0b9e-44b6-9619-93b88cf3a501-kube-api-access-cdm8b\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.891853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891684 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7135034a-0b9e-44b6-9619-93b88cf3a501-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.891853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.891728 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:42.992577 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992536 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-installation-pull-secrets\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.992756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992596 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxj4w\" (UniqueName: \"kubernetes.io/projected/30ad9c8e-526a-4cbd-973d-8719f34ead2e-kube-api-access-xxj4w\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:42.992756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992624 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7135034a-0b9e-44b6-9619-93b88cf3a501-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.992756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992650 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7skm4\" (UniqueName: \"kubernetes.io/projected/37b281f4-5dcc-4677-82fd-458213d77cb1-kube-api-access-7skm4\") pod \"network-check-source-7bc5b6bf85-hz8nf\" (UID: \"37b281f4-5dcc-4677-82fd-458213d77cb1\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" Jul 21 12:11:42.992756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992728 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrr9w\" (UniqueName: \"kubernetes.io/projected/41a6c18f-f386-4739-8fa7-2acdfdf46223-kube-api-access-zrr9w\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:42.992975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992774 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfr8g\" (UniqueName: \"kubernetes.io/projected/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-kube-api-access-dfr8g\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.992975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992806 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a380abd7-3ed2-4c49-b8b7-18431a1f7469-snapshots\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.992975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992833 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a380abd7-3ed2-4c49-b8b7-18431a1f7469-service-ca-bundle\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.992975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992857 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:42.992975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992886 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzbdk\" (UniqueName: \"kubernetes.io/projected/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-kube-api-access-zzbdk\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:42.992975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992943 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdfgl\" (UniqueName: \"kubernetes.io/projected/67b67449-f536-4272-bb03-f8224c8e196e-kube-api-access-gdfgl\") pod \"volume-data-source-validator-78cf4c6f6-8hg6z\" (UID: \"67b67449-f536-4272-bb03-f8224c8e196e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.992982 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993010 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-trusted-ca\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993043 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-image-registry-private-configuration\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.993089 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993101 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khxh8\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-kube-api-access-khxh8\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993132 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-certificates\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993158 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-config\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993180 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396d16ec-f45b-427e-846a-68ee756f846c-tmp-dir\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993208 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdm8b\" (UniqueName: \"kubernetes.io/projected/7135034a-0b9e-44b6-9619-93b88cf3a501-kube-api-access-cdm8b\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.993247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993234 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z76l\" (UniqueName: \"kubernetes.io/projected/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-kube-api-access-9z76l\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993257 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a380abd7-3ed2-4c49-b8b7-18431a1f7469-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.993105 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c848ff744-k8fkq: secret "image-registry-tls" not found Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993288 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993336 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbqh2\" (UniqueName: \"kubernetes.io/projected/a380abd7-3ed2-4c49-b8b7-18431a1f7469-kube-api-access-tbqh2\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993371 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.993415 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls podName:572604ec-8a1f-45fc-a6c6-26e661e4ff7d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.493395255 +0000 UTC m=+36.987224225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls") pod "image-registry-6c848ff744-k8fkq" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d") : secret "image-registry-tls" not found Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993451 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-config\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993476 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-serving-cert\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993500 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/396d16ec-f45b-427e-846a-68ee756f846c-config-volume\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993550 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/317bf622-f052-40df-be20-3269ccdf1cda-cert\") pod \"ingress-canary-fl9m4\" (UID: \"317bf622-f052-40df-be20-3269ccdf1cda\") " pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993575 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl5k8\" (UniqueName: \"kubernetes.io/projected/396d16ec-f45b-427e-846a-68ee756f846c-kube-api-access-nl5k8\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993604 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41a6c18f-f386-4739-8fa7-2acdfdf46223-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:42.993959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993655 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.993676 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993684 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74mcv\" (UniqueName: \"kubernetes.io/projected/317bf622-f052-40df-be20-3269ccdf1cda-kube-api-access-74mcv\") pod \"ingress-canary-fl9m4\" (UID: \"317bf622-f052-40df-be20-3269ccdf1cda\") " pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993714 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-default-certificate\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-ca-trust-extracted\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993776 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-certificates\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993797 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-stats-auth\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:42.993839 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls podName:30ad9c8e-526a-4cbd-973d-8719f34ead2e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.49380545 +0000 UTC m=+36.987634427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gqnh2" (UID: "30ad9c8e-526a-4cbd-973d-8719f34ead2e") : secret "samples-operator-tls" not found Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a380abd7-3ed2-4c49-b8b7-18431a1f7469-serving-cert\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993916 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-serving-cert\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993940 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.993980 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-trusted-ca\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994006 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:42.994526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994037 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-bound-sa-token\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.995161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994063 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7135034a-0b9e-44b6-9619-93b88cf3a501-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.995161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994088 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:42.995161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994111 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a380abd7-3ed2-4c49-b8b7-18431a1f7469-tmp\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:42.995161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994111 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-ca-trust-extracted\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.995161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994267 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-config\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:42.995161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.994957 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7135034a-0b9e-44b6-9619-93b88cf3a501-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.995544 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.995497 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-trusted-ca\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.998145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.997959 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-installation-pull-secrets\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.998145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.997959 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-image-registry-private-configuration\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:42.998305 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.998181 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7135034a-0b9e-44b6-9619-93b88cf3a501-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:42.998671 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:42.998586 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-serving-cert\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:43.003068 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.003011 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxj4w\" (UniqueName: \"kubernetes.io/projected/30ad9c8e-526a-4cbd-973d-8719f34ead2e-kube-api-access-xxj4w\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:43.003860 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.003821 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-bound-sa-token\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:43.004014 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.003993 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfr8g\" (UniqueName: \"kubernetes.io/projected/05aaecb6-6e63-4345-b3a3-4a930d3ef9bd-kube-api-access-dfr8g\") pod \"service-ca-operator-5bcbc94c78-7hhmp\" (UID: \"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:43.004937 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.004908 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdfgl\" (UniqueName: \"kubernetes.io/projected/67b67449-f536-4272-bb03-f8224c8e196e-kube-api-access-gdfgl\") pod \"volume-data-source-validator-78cf4c6f6-8hg6z\" (UID: \"67b67449-f536-4272-bb03-f8224c8e196e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" Jul 21 12:11:43.005371 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.005347 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdm8b\" (UniqueName: \"kubernetes.io/projected/7135034a-0b9e-44b6-9619-93b88cf3a501-kube-api-access-cdm8b\") pod \"kube-storage-version-migrator-operator-7658d9b749-rgkhb\" (UID: \"7135034a-0b9e-44b6-9619-93b88cf3a501\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:43.005947 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.005921 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khxh8\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-kube-api-access-khxh8\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:43.094882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.094800 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7skm4\" (UniqueName: \"kubernetes.io/projected/37b281f4-5dcc-4677-82fd-458213d77cb1-kube-api-access-7skm4\") pod \"network-check-source-7bc5b6bf85-hz8nf\" (UID: \"37b281f4-5dcc-4677-82fd-458213d77cb1\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" Jul 21 12:11:43.094882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.094842 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrr9w\" (UniqueName: \"kubernetes.io/projected/41a6c18f-f386-4739-8fa7-2acdfdf46223-kube-api-access-zrr9w\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:43.094882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.094866 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a380abd7-3ed2-4c49-b8b7-18431a1f7469-snapshots\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.094882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.094885 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a380abd7-3ed2-4c49-b8b7-18431a1f7469-service-ca-bundle\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.095165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.094906 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.095165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.094929 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzbdk\" (UniqueName: \"kubernetes.io/projected/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-kube-api-access-zzbdk\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.095165 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.095061 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.595038041 +0000 UTC m=+37.088867032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : configmap references non-existent config key: service-ca.crt Jul 21 12:11:43.095165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095129 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-trusted-ca\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.095308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095172 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-config\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.095308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396d16ec-f45b-427e-846a-68ee756f846c-tmp-dir\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.095308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095226 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z76l\" (UniqueName: \"kubernetes.io/projected/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-kube-api-access-9z76l\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.095308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095246 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a380abd7-3ed2-4c49-b8b7-18431a1f7469-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.095308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095270 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:43.095308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095293 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbqh2\" (UniqueName: \"kubernetes.io/projected/a380abd7-3ed2-4c49-b8b7-18431a1f7469-kube-api-access-tbqh2\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095330 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-serving-cert\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095350 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/396d16ec-f45b-427e-846a-68ee756f846c-config-volume\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095389 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/317bf622-f052-40df-be20-3269ccdf1cda-cert\") pod \"ingress-canary-fl9m4\" (UID: \"317bf622-f052-40df-be20-3269ccdf1cda\") " pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095413 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl5k8\" (UniqueName: \"kubernetes.io/projected/396d16ec-f45b-427e-846a-68ee756f846c-kube-api-access-nl5k8\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095462 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41a6c18f-f386-4739-8fa7-2acdfdf46223-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095488 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095512 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74mcv\" (UniqueName: \"kubernetes.io/projected/317bf622-f052-40df-be20-3269ccdf1cda-kube-api-access-74mcv\") pod \"ingress-canary-fl9m4\" (UID: \"317bf622-f052-40df-be20-3269ccdf1cda\") " pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:43.095554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095537 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-default-certificate\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095568 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-stats-auth\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095578 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/396d16ec-f45b-427e-846a-68ee756f846c-tmp-dir\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095589 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a380abd7-3ed2-4c49-b8b7-18431a1f7469-serving-cert\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095602 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a380abd7-3ed2-4c49-b8b7-18431a1f7469-service-ca-bundle\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095611 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095649 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.095713 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a380abd7-3ed2-4c49-b8b7-18431a1f7469-snapshots\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.095764 2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.095809 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls podName:41a6c18f-f386-4739-8fa7-2acdfdf46223 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.595795199 +0000 UTC m=+37.089624175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-7l28l" (UID: "41a6c18f-f386-4739-8fa7-2acdfdf46223") : secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:43.095868 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.095864 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:11:43.096445 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.095896 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.595884703 +0000 UTC m=+37.089713678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : secret "router-metrics-certs-default" not found Jul 21 12:11:43.096445 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.096103 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/396d16ec-f45b-427e-846a-68ee756f846c-config-volume\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.096445 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.096142 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-trusted-ca\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.096635 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.096455 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 12:11:43.096635 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.096507 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert podName:133dbdd2-a362-4d8c-adfd-fd1dca5ff65d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.596492367 +0000 UTC m=+37.090321339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-22p7z" (UID: "133dbdd2-a362-4d8c-adfd-fd1dca5ff65d") : secret "networking-console-plugin-cert" not found Jul 21 12:11:43.096635 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.096538 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a380abd7-3ed2-4c49-b8b7-18431a1f7469-tmp\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.096635 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.096573 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-config\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.096876 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.096729 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:11:43.096876 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.096771 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls podName:396d16ec-f45b-427e-846a-68ee756f846c nodeName:}" failed. No retries permitted until 2026-07-21 12:11:43.596758151 +0000 UTC m=+37.090587133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls") pod "dns-default-nrn6d" (UID: "396d16ec-f45b-427e-846a-68ee756f846c") : secret "dns-default-metrics-tls" not found Jul 21 12:11:43.096990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.096872 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41a6c18f-f386-4739-8fa7-2acdfdf46223-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:43.097688 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.097272 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:43.097688 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.097346 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a380abd7-3ed2-4c49-b8b7-18431a1f7469-tmp\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.097862 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.097816 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a380abd7-3ed2-4c49-b8b7-18431a1f7469-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.098408 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.098370 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a380abd7-3ed2-4c49-b8b7-18431a1f7469-serving-cert\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.098508 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.098377 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-serving-cert\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.098906 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.098886 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-default-certificate\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.098999 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.098908 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-stats-auth\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.099113 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.099095 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/317bf622-f052-40df-be20-3269ccdf1cda-cert\") pod \"ingress-canary-fl9m4\" (UID: \"317bf622-f052-40df-be20-3269ccdf1cda\") " pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:43.107631 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.107582 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzbdk\" (UniqueName: \"kubernetes.io/projected/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-kube-api-access-zzbdk\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.107631 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.107605 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7skm4\" (UniqueName: \"kubernetes.io/projected/37b281f4-5dcc-4677-82fd-458213d77cb1-kube-api-access-7skm4\") pod \"network-check-source-7bc5b6bf85-hz8nf\" (UID: \"37b281f4-5dcc-4677-82fd-458213d77cb1\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" Jul 21 12:11:43.107631 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.107612 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl5k8\" (UniqueName: \"kubernetes.io/projected/396d16ec-f45b-427e-846a-68ee756f846c-kube-api-access-nl5k8\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.108127 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.108109 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbqh2\" (UniqueName: \"kubernetes.io/projected/a380abd7-3ed2-4c49-b8b7-18431a1f7469-kube-api-access-tbqh2\") pod \"insights-operator-555d9d7885-zsqww\" (UID: \"a380abd7-3ed2-4c49-b8b7-18431a1f7469\") " pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.108521 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.108498 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74mcv\" (UniqueName: \"kubernetes.io/projected/317bf622-f052-40df-be20-3269ccdf1cda-kube-api-access-74mcv\") pod \"ingress-canary-fl9m4\" (UID: \"317bf622-f052-40df-be20-3269ccdf1cda\") " pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:43.109059 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.109041 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrr9w\" (UniqueName: \"kubernetes.io/projected/41a6c18f-f386-4739-8fa7-2acdfdf46223-kube-api-access-zrr9w\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:43.109606 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.109580 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z76l\" (UniqueName: \"kubernetes.io/projected/4fc860f4-873b-4e1e-8e3c-503f24ab6d86-kube-api-access-9z76l\") pod \"console-operator-565b7b569d-dvskp\" (UID: \"4fc860f4-873b-4e1e-8e3c-503f24ab6d86\") " pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.111794 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.111780 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" Jul 21 12:11:43.132601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.132568 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" Jul 21 12:11:43.154796 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.154756 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" Jul 21 12:11:43.173668 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.173593 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-zsqww" Jul 21 12:11:43.189688 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.189227 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:11:43.212360 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.211656 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" Jul 21 12:11:43.219127 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.219098 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fl9m4" Jul 21 12:11:43.327254 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.326795 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp"] Jul 21 12:11:43.330209 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.329187 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z"] Jul 21 12:11:43.337740 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:43.337706 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b67449_f536_4272_bb03_f8224c8e196e.slice/crio-79ca1886279f7ce681fbb43bdefad3c9e5dc439cce44f96fc6ae91330480da20 WatchSource:0}: Error finding container 79ca1886279f7ce681fbb43bdefad3c9e5dc439cce44f96fc6ae91330480da20: Status 404 returned error can't find the container with id 79ca1886279f7ce681fbb43bdefad3c9e5dc439cce44f96fc6ae91330480da20 Jul 21 12:11:43.390552 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.390519 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-zsqww"] Jul 21 12:11:43.394914 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:43.394232 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda380abd7_3ed2_4c49_b8b7_18431a1f7469.slice/crio-8088a377eed2a535de9bd12bcbaa04f9911ca060a42e3010208ac35a5fb4ab4f WatchSource:0}: Error finding container 8088a377eed2a535de9bd12bcbaa04f9911ca060a42e3010208ac35a5fb4ab4f: Status 404 returned error can't find the container with id 8088a377eed2a535de9bd12bcbaa04f9911ca060a42e3010208ac35a5fb4ab4f Jul 21 12:11:43.395995 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.395970 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb"] Jul 21 12:11:43.403942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.403913 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-dvskp"] Jul 21 12:11:43.409178 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:43.409153 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc860f4_873b_4e1e_8e3c_503f24ab6d86.slice/crio-6bb0947444987b7243b6edeff624e2db34fb5ab6ffce4a0bd04566b17a52474a WatchSource:0}: Error finding container 6bb0947444987b7243b6edeff624e2db34fb5ab6ffce4a0bd04566b17a52474a: Status 404 returned error can't find the container with id 6bb0947444987b7243b6edeff624e2db34fb5ab6ffce4a0bd04566b17a52474a Jul 21 12:11:43.435767 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.435735 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fl9m4"] Jul 21 12:11:43.439164 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:43.439137 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317bf622_f052_40df_be20_3269ccdf1cda.slice/crio-ce8e8384e81f0f6167c14a05992234b0cda84741137f8d44d737be5a3c9b0752 WatchSource:0}: Error finding container ce8e8384e81f0f6167c14a05992234b0cda84741137f8d44d737be5a3c9b0752: Status 404 returned error can't find the container with id ce8e8384e81f0f6167c14a05992234b0cda84741137f8d44d737be5a3c9b0752 Jul 21 12:11:43.454570 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.454502 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf"] Jul 21 12:11:43.460312 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:11:43.460281 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b281f4_5dcc_4677_82fd_458213d77cb1.slice/crio-610f66c976c084982b887941095e9a80afa5da1877d994e48ab79ff2ae3292b8 WatchSource:0}: Error finding container 610f66c976c084982b887941095e9a80afa5da1877d994e48ab79ff2ae3292b8: Status 404 returned error can't find the container with id 610f66c976c084982b887941095e9a80afa5da1877d994e48ab79ff2ae3292b8 Jul 21 12:11:43.502229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.502200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:43.502392 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.502276 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:43.502392 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.502367 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:11:43.502392 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.502388 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c848ff744-k8fkq: secret "image-registry-tls" not found Jul 21 12:11:43.502536 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.502443 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 12:11:43.502536 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.502469 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls podName:572604ec-8a1f-45fc-a6c6-26e661e4ff7d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.502446042 +0000 UTC m=+37.996275027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls") pod "image-registry-6c848ff744-k8fkq" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d") : secret "image-registry-tls" not found Jul 21 12:11:43.502536 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.502501 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls podName:30ad9c8e-526a-4cbd-973d-8719f34ead2e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.502483988 +0000 UTC m=+37.996312965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gqnh2" (UID: "30ad9c8e-526a-4cbd-973d-8719f34ead2e") : secret "samples-operator-tls" not found Jul 21 12:11:43.603878 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.603789 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:43.603878 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.603846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:43.603878 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.603877 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.603901 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:43.603943 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.603944 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604008 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert podName:133dbdd2-a362-4d8c-adfd-fd1dca5ff65d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.603992506 +0000 UTC m=+38.097821480 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-22p7z" (UID: "133dbdd2-a362-4d8c-adfd-fd1dca5ff65d") : secret "networking-console-plugin-cert" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604009 2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604052 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604063 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls podName:41a6c18f-f386-4739-8fa7-2acdfdf46223 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.604050477 +0000 UTC m=+38.097879451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-7l28l" (UID: "41a6c18f-f386-4739-8fa7-2acdfdf46223") : secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604009 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604097 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.60408374 +0000 UTC m=+38.097912714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : secret "router-metrics-certs-default" not found Jul 21 12:11:43.604138 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604144 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls podName:396d16ec-f45b-427e-846a-68ee756f846c nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.604129965 +0000 UTC m=+38.097958951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls") pod "dns-default-nrn6d" (UID: "396d16ec-f45b-427e-846a-68ee756f846c") : secret "dns-default-metrics-tls" not found Jul 21 12:11:43.604546 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:43.604167 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:44.604158446 +0000 UTC m=+38.097987417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : configmap references non-existent config key: service-ca.crt Jul 21 12:11:44.114501 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.114459 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:11:44.115491 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.114936 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:11:44.115491 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.115390 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:11:44.117647 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.117621 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:11:44.118124 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.118107 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2w9zn\"" Jul 21 12:11:44.118338 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.118321 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:11:44.118578 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.118560 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8v447\"" Jul 21 12:11:44.289876 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.289831 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" event={"ID":"7135034a-0b9e-44b6-9619-93b88cf3a501","Type":"ContainerStarted","Data":"2a3c3ae661a2a7aeec941e4b8eb78a6aad5d96730886865478203c2e06d5453c"} Jul 21 12:11:44.296642 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.296564 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" event={"ID":"37b281f4-5dcc-4677-82fd-458213d77cb1","Type":"ContainerStarted","Data":"610f66c976c084982b887941095e9a80afa5da1877d994e48ab79ff2ae3292b8"} Jul 21 12:11:44.310039 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.310007 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" event={"ID":"4fc860f4-873b-4e1e-8e3c-503f24ab6d86","Type":"ContainerStarted","Data":"6bb0947444987b7243b6edeff624e2db34fb5ab6ffce4a0bd04566b17a52474a"} Jul 21 12:11:44.311934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.311905 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" event={"ID":"67b67449-f536-4272-bb03-f8224c8e196e","Type":"ContainerStarted","Data":"79ca1886279f7ce681fbb43bdefad3c9e5dc439cce44f96fc6ae91330480da20"} Jul 21 12:11:44.313669 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.313599 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fl9m4" event={"ID":"317bf622-f052-40df-be20-3269ccdf1cda","Type":"ContainerStarted","Data":"ce8e8384e81f0f6167c14a05992234b0cda84741137f8d44d737be5a3c9b0752"} Jul 21 12:11:44.316671 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.316614 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-zsqww" event={"ID":"a380abd7-3ed2-4c49-b8b7-18431a1f7469","Type":"ContainerStarted","Data":"8088a377eed2a535de9bd12bcbaa04f9911ca060a42e3010208ac35a5fb4ab4f"} Jul 21 12:11:44.320126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.320069 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" event={"ID":"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd","Type":"ContainerStarted","Data":"b8c9781e88653dae2bb0748cb13e952053a087d9c12f40984aab79964a41ff71"} Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.513534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.513624 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.513861 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.513933 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls podName:30ad9c8e-526a-4cbd-973d-8719f34ead2e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.513914765 +0000 UTC m=+40.007743741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gqnh2" (UID: "30ad9c8e-526a-4cbd-973d-8719f34ead2e") : secret "samples-operator-tls" not found Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.514355 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.514372 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c848ff744-k8fkq: secret "image-registry-tls" not found Jul 21 12:11:44.514468 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.514416 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls podName:572604ec-8a1f-45fc-a6c6-26e661e4ff7d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.514401916 +0000 UTC m=+40.008230890 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls") pod "image-registry-6c848ff744-k8fkq" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d") : secret "image-registry-tls" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.615057 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.615151 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.615245 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.615310 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:44.615351 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.615516 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.615595 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls podName:396d16ec-f45b-427e-846a-68ee756f846c nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.615575342 +0000 UTC m=+40.109404316 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls") pod "dns-default-nrn6d" (UID: "396d16ec-f45b-427e-846a-68ee756f846c") : secret "dns-default-metrics-tls" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.615972 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.616021 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.61600664 +0000 UTC m=+40.109835613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : secret "router-metrics-certs-default" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.616086 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.616075656 +0000 UTC m=+40.109904630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : configmap references non-existent config key: service-ca.crt Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.616135 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.616163 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert podName:133dbdd2-a362-4d8c-adfd-fd1dca5ff65d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.616153668 +0000 UTC m=+40.109982641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-22p7z" (UID: "133dbdd2-a362-4d8c-adfd-fd1dca5ff65d") : secret "networking-console-plugin-cert" not found Jul 21 12:11:44.616285 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.616213 2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:44.617139 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:44.616244 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls podName:41a6c18f-f386-4739-8fa7-2acdfdf46223 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:46.616234535 +0000 UTC m=+40.110063509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-7l28l" (UID: "41a6c18f-f386-4739-8fa7-2acdfdf46223") : secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:46.537123 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.536925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:46.537627 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.537163 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:46.537627 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.537059 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:11:46.537627 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.537215 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c848ff744-k8fkq: secret "image-registry-tls" not found Jul 21 12:11:46.537627 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.537280 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls podName:572604ec-8a1f-45fc-a6c6-26e661e4ff7d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.537259062 +0000 UTC m=+44.031088046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls") pod "image-registry-6c848ff744-k8fkq" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d") : secret "image-registry-tls" not found Jul 21 12:11:46.537627 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.537328 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 12:11:46.537627 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.537377 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls podName:30ad9c8e-526a-4cbd-973d-8719f34ead2e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.537363605 +0000 UTC m=+44.031192608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gqnh2" (UID: "30ad9c8e-526a-4cbd-973d-8719f34ead2e") : secret "samples-operator-tls" not found Jul 21 12:11:46.638515 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.638478 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:46.638691 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.638538 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:46.638691 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638630 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:11:46.638691 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638657 2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:46.638862 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.638685 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:46.638862 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638697 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls podName:396d16ec-f45b-427e-846a-68ee756f846c nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.638678807 +0000 UTC m=+44.132507783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls") pod "dns-default-nrn6d" (UID: "396d16ec-f45b-427e-846a-68ee756f846c") : secret "dns-default-metrics-tls" not found Jul 21 12:11:46.638862 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638730 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls podName:41a6c18f-f386-4739-8fa7-2acdfdf46223 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.638713852 +0000 UTC m=+44.132542836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-7l28l" (UID: "41a6c18f-f386-4739-8fa7-2acdfdf46223") : secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:46.638862 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638801 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:11:46.638862 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638858 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.638845091 +0000 UTC m=+44.132674068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : secret "router-metrics-certs-default" not found Jul 21 12:11:46.638862 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.638803 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:46.639106 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.638876 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.638867306 +0000 UTC m=+44.132696277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : configmap references non-existent config key: service-ca.crt Jul 21 12:11:46.639106 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:46.638972 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:46.639106 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.639084 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 12:11:46.639203 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:46.639119 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert podName:133dbdd2-a362-4d8c-adfd-fd1dca5ff65d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:50.639108091 +0000 UTC m=+44.132937063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-22p7z" (UID: "133dbdd2-a362-4d8c-adfd-fd1dca5ff65d") : secret "networking-console-plugin-cert" not found Jul 21 12:11:50.575236 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.575194 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:50.575714 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.575259 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:50.575714 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.575361 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:11:50.575714 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.575381 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c848ff744-k8fkq: secret "image-registry-tls" not found Jul 21 12:11:50.575714 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.575417 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 12:11:50.575714 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.575460 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls podName:572604ec-8a1f-45fc-a6c6-26e661e4ff7d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.575443992 +0000 UTC m=+52.069272963 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls") pod "image-registry-6c848ff744-k8fkq" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d") : secret "image-registry-tls" not found Jul 21 12:11:50.575714 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.575480 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls podName:30ad9c8e-526a-4cbd-973d-8719f34ead2e nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.575467716 +0000 UTC m=+52.069296692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gqnh2" (UID: "30ad9c8e-526a-4cbd-973d-8719f34ead2e") : secret "samples-operator-tls" not found Jul 21 12:11:50.675968 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.675932 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:50.676136 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.676026 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:50.676136 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.676083 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:50.676136 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676118 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.676098823 +0000 UTC m=+52.169927796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : configmap references non-existent config key: service-ca.crt Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.676158 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676167 2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676168 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:50.676200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676208 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls podName:41a6c18f-f386-4739-8fa7-2acdfdf46223 nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.676195917 +0000 UTC m=+52.170024903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-7l28l" (UID: "41a6c18f-f386-4739-8fa7-2acdfdf46223") : secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676236 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert podName:133dbdd2-a362-4d8c-adfd-fd1dca5ff65d nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.676217667 +0000 UTC m=+52.170046656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-22p7z" (UID: "133dbdd2-a362-4d8c-adfd-fd1dca5ff65d") : secret "networking-console-plugin-cert" not found Jul 21 12:11:50.676271 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676256 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:11:50.676625 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676289 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.676282357 +0000 UTC m=+52.170111327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : secret "router-metrics-certs-default" not found Jul 21 12:11:50.676625 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676302 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:11:50.676625 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:50.676371 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls podName:396d16ec-f45b-427e-846a-68ee756f846c nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.676354706 +0000 UTC m=+52.170183691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls") pod "dns-default-nrn6d" (UID: "396d16ec-f45b-427e-846a-68ee756f846c") : secret "dns-default-metrics-tls" not found Jul 21 12:11:53.350479 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.350441 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" event={"ID":"67b67449-f536-4272-bb03-f8224c8e196e","Type":"ContainerStarted","Data":"9e31af8cc8351a7999a558077ba50017561b1f541742f22cb7788df49becdf12"} Jul 21 12:11:53.353589 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.353094 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fl9m4" event={"ID":"317bf622-f052-40df-be20-3269ccdf1cda","Type":"ContainerStarted","Data":"5ae571c373636a3f039e9293c13a8d60191e46bad463a4389465b959766d4014"} Jul 21 12:11:53.356365 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.356340 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-zsqww" event={"ID":"a380abd7-3ed2-4c49-b8b7-18431a1f7469","Type":"ContainerStarted","Data":"182407b0e277ea923e0e5f3ff62237fa001196b49141562ee52eae8e2e519ca6"} Jul 21 12:11:53.358203 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.358176 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" event={"ID":"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd","Type":"ContainerStarted","Data":"df7c7daae8904dff41585dd9f58ec5d28f65e1c6d696e60b33ef60de84c4eca0"} Jul 21 12:11:53.362236 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.362213 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/0.log" Jul 21 12:11:53.362337 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.362249 2550 generic.go:356] "Generic (PLEG): container finished" podID="4fc860f4-873b-4e1e-8e3c-503f24ab6d86" containerID="722db539858ab0d185836db437155d415c96a688122fe4a65fa4fb9222ad3bb9" exitCode=255 Jul 21 12:11:53.362337 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.362280 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" event={"ID":"4fc860f4-873b-4e1e-8e3c-503f24ab6d86","Type":"ContainerDied","Data":"722db539858ab0d185836db437155d415c96a688122fe4a65fa4fb9222ad3bb9"} Jul 21 12:11:53.362576 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.362560 2550 scope.go:117] "RemoveContainer" containerID="722db539858ab0d185836db437155d415c96a688122fe4a65fa4fb9222ad3bb9" Jul 21 12:11:53.369629 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.369577 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-8hg6z" podStartSLOduration=17.854058514 podStartE2EDuration="27.369564003s" totalImagesPullingTime="9.515505489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:53.368177911 +0000 UTC m=+46.862006905" watchObservedRunningTime="2026-07-21 12:11:53.369564003 +0000 UTC m=+46.863392996" Jul 21 12:11:53.386444 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.386372 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fl9m4" podStartSLOduration=1.9718415980000001 podStartE2EDuration="11.386348942s" totalImagesPullingTime="9.414507344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:53.385326633 +0000 UTC m=+46.879155628" watchObservedRunningTime="2026-07-21 12:11:53.386348942 +0000 UTC m=+46.880177936" Jul 21 12:11:53.406664 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.406610 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-zsqww" podStartSLOduration=17.946990598 podStartE2EDuration="27.406589501s" totalImagesPullingTime="9.459598903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:53.405199784 +0000 UTC m=+46.899028779" watchObservedRunningTime="2026-07-21 12:11:53.406589501 +0000 UTC m=+46.900418495" Jul 21 12:11:53.426218 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:53.425222 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" podStartSLOduration=17.905130474 podStartE2EDuration="27.425201395s" totalImagesPullingTime="9.520070921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:53.423982239 +0000 UTC m=+46.917811234" watchObservedRunningTime="2026-07-21 12:11:53.425201395 +0000 UTC m=+46.919030389" Jul 21 12:11:54.371275 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.371236 2550 generic.go:356] "Generic (PLEG): container finished" podID="91b0a1c8-c980-4d55-b819-a391a6a27fb7" containerID="4fc62c458ad70757d6d7c6fe73661962bfb63d49e454fd10719ad49c97a2dcfa" exitCode=0 Jul 21 12:11:54.371765 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.371330 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerDied","Data":"4fc62c458ad70757d6d7c6fe73661962bfb63d49e454fd10719ad49c97a2dcfa"} Jul 21 12:11:54.373015 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.372989 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" event={"ID":"7135034a-0b9e-44b6-9619-93b88cf3a501","Type":"ContainerStarted","Data":"4c2fe14892f51e6689d8e74b1856f245c6f371144c0875e66c8e84543f15e69f"} Jul 21 12:11:54.374595 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.374572 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" event={"ID":"37b281f4-5dcc-4677-82fd-458213d77cb1","Type":"ContainerStarted","Data":"5c4ad88c70c85320713cd980628cb8439a6825cb03459c292753aa83ecd1a65e"} Jul 21 12:11:54.378526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.377105 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:11:54.378526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.377512 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/0.log" Jul 21 12:11:54.378526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.377539 2550 generic.go:356] "Generic (PLEG): container finished" podID="4fc860f4-873b-4e1e-8e3c-503f24ab6d86" containerID="143480cd7a8597ca8e2a44acd20fa05eba4691420d0c1e91dabf015cfc4986d0" exitCode=255 Jul 21 12:11:54.378526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.378169 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" event={"ID":"4fc860f4-873b-4e1e-8e3c-503f24ab6d86","Type":"ContainerDied","Data":"143480cd7a8597ca8e2a44acd20fa05eba4691420d0c1e91dabf015cfc4986d0"} Jul 21 12:11:54.378526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.378210 2550 scope.go:117] "RemoveContainer" containerID="722db539858ab0d185836db437155d415c96a688122fe4a65fa4fb9222ad3bb9" Jul 21 12:11:54.378526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.378419 2550 scope.go:117] "RemoveContainer" containerID="143480cd7a8597ca8e2a44acd20fa05eba4691420d0c1e91dabf015cfc4986d0" Jul 21 12:11:54.378865 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:54.378579 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-dvskp_openshift-console-operator(4fc860f4-873b-4e1e-8e3c-503f24ab6d86)\"" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" podUID="4fc860f4-873b-4e1e-8e3c-503f24ab6d86" Jul 21 12:11:54.418930 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.418870 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-hz8nf" podStartSLOduration=19.012987573 podStartE2EDuration="28.418849188s" totalImagesPullingTime="9.405861615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:54.416803014 +0000 UTC m=+47.910632008" watchObservedRunningTime="2026-07-21 12:11:54.418849188 +0000 UTC m=+47.912678182" Jul 21 12:11:54.458929 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.458861 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" podStartSLOduration=18.564509746 podStartE2EDuration="28.458822315s" totalImagesPullingTime="9.894312569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:54.458359311 +0000 UTC m=+47.952188306" watchObservedRunningTime="2026-07-21 12:11:54.458822315 +0000 UTC m=+47.952651309" Jul 21 12:11:54.809079 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.809038 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4mjkh"] Jul 21 12:11:54.833272 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.833242 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4mjkh"] Jul 21 12:11:54.833526 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.833394 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:54.836922 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.836344 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2gbfg\"" Jul 21 12:11:54.836922 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.836570 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 12:11:54.836922 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.836789 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 12:11:54.922861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.922821 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:54.923031 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.922925 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:54.923031 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.922951 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f1663b6-40b5-4515-b534-041eb295876f-data-volume\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:54.923031 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.922967 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fq27\" (UniqueName: \"kubernetes.io/projected/1f1663b6-40b5-4515-b534-041eb295876f-kube-api-access-9fq27\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:54.923031 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:54.923000 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f1663b6-40b5-4515-b534-041eb295876f-crio-socket\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.023750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.023716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.023915 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.023799 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.023915 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.023824 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f1663b6-40b5-4515-b534-041eb295876f-data-volume\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.023915 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.023839 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fq27\" (UniqueName: \"kubernetes.io/projected/1f1663b6-40b5-4515-b534-041eb295876f-kube-api-access-9fq27\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.023915 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.023862 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f1663b6-40b5-4515-b534-041eb295876f-crio-socket\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.023915 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.023880 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:11:55.024154 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.023967 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:55.523945923 +0000 UTC m=+49.017774919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : configmap "kube-rbac-proxy" not found Jul 21 12:11:55.024154 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.023986 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:11:55.024154 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.024041 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:55.524024501 +0000 UTC m=+49.017853484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:11:55.024154 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.024054 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f1663b6-40b5-4515-b534-041eb295876f-crio-socket\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.024308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.024214 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f1663b6-40b5-4515-b534-041eb295876f-data-volume\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.036622 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.036592 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fq27\" (UniqueName: \"kubernetes.io/projected/1f1663b6-40b5-4515-b534-041eb295876f-kube-api-access-9fq27\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.383097 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.383069 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:11:55.383538 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.383455 2550 scope.go:117] "RemoveContainer" containerID="143480cd7a8597ca8e2a44acd20fa05eba4691420d0c1e91dabf015cfc4986d0" Jul 21 12:11:55.383655 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.383627 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-dvskp_openshift-console-operator(4fc860f4-873b-4e1e-8e3c-503f24ab6d86)\"" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" podUID="4fc860f4-873b-4e1e-8e3c-503f24ab6d86" Jul 21 12:11:55.386115 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.386087 2550 generic.go:356] "Generic (PLEG): container finished" podID="91b0a1c8-c980-4d55-b819-a391a6a27fb7" containerID="846bc96ad6a8583bbe0aa7e8a81f3a4be9da2db910c5cb9664a8d1f1cbe638de" exitCode=0 Jul 21 12:11:55.386247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.386165 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerDied","Data":"846bc96ad6a8583bbe0aa7e8a81f3a4be9da2db910c5cb9664a8d1f1cbe638de"} Jul 21 12:11:55.528075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.528046 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.528159 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:55.528150 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:55.528202 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.528159 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:11:55.528237 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.528213 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:56.52819645 +0000 UTC m=+50.022025423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : configmap "kube-rbac-proxy" not found Jul 21 12:11:55.528275 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.528244 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:11:55.528306 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:55.528285 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:56.528272069 +0000 UTC m=+50.022101039 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:11:56.393821 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:56.393788 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" event={"ID":"91b0a1c8-c980-4d55-b819-a391a6a27fb7","Type":"ContainerStarted","Data":"2f71dda4e1c8cdc212c8517d7757dd62461519863d8f6b92316b7431ec35d0d5"} Jul 21 12:11:56.416372 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:56.416323 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7k4bk" podStartSLOduration=8.995258237 podStartE2EDuration="49.416304413s" totalImagesPullingTime="40.421046176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:11:56.416091682 +0000 UTC m=+49.909920675" watchObservedRunningTime="2026-07-21 12:11:56.416304413 +0000 UTC m=+49.910133408" Jul 21 12:11:56.540697 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:56.540654 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:56.540861 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:56.540775 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:11:56.540861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:56.540797 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:56.540861 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:56.540833 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.540817741 +0000 UTC m=+52.034646712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : configmap "kube-rbac-proxy" not found Jul 21 12:11:56.540973 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:56.540901 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:11:56.540973 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:56.540960 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:11:58.540944081 +0000 UTC m=+52.034773064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:11:57.008144 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:57.008114 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pb5qj_502316a9-ade5-4bf4-b1f8-f6868b87987b/dns-node-resolver/0.log" Jul 21 12:11:58.006346 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.006323 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wxzfm_a5f7f769-70f8-4cfe-a810-ff6d3e476ce4/node-ca/0.log" Jul 21 12:11:58.406748 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.406676 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fl9m4_317bf622-f052-40df-be20-3269ccdf1cda/serve-healthcheck-canary/0.log" Jul 21 12:11:58.556581 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.556549 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:58.556779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.556639 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:11:58.556779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.556679 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:11:58.556779 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.556746 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:12:02.556725171 +0000 UTC m=+56.050554153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : configmap "kube-rbac-proxy" not found Jul 21 12:11:58.556947 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.556785 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:11:58.556947 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.556840 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:12:02.556825409 +0000 UTC m=+56.050654420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:11:58.657494 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.657391 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:11:58.657643 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.657562 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:11:58.657643 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.657589 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c848ff744-k8fkq: secret "image-registry-tls" not found Jul 21 12:11:58.657643 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.657588 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:11:58.657776 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.657647 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls podName:572604ec-8a1f-45fc-a6c6-26e661e4ff7d nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.657628708 +0000 UTC m=+68.151457714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls") pod "image-registry-6c848ff744-k8fkq" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d") : secret "image-registry-tls" not found Jul 21 12:11:58.657776 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.657688 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 12:11:58.657776 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.657731 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls podName:30ad9c8e-526a-4cbd-973d-8719f34ead2e nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.657720953 +0000 UTC m=+68.151549930 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gqnh2" (UID: "30ad9c8e-526a-4cbd-973d-8719f34ead2e") : secret "samples-operator-tls" not found Jul 21 12:11:58.758824 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.758792 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:11:58.758994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.758838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:11:58.758994 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.758929 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:11:58.758994 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.758947 2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:58.758994 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.758972 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls podName:396d16ec-f45b-427e-846a-68ee756f846c nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.758959332 +0000 UTC m=+68.252788303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls") pod "dns-default-nrn6d" (UID: "396d16ec-f45b-427e-846a-68ee756f846c") : secret "dns-default-metrics-tls" not found Jul 21 12:11:58.758994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.758970 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.759001 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls podName:41a6c18f-f386-4739-8fa7-2acdfdf46223 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.758988394 +0000 UTC m=+68.252817365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-7l28l" (UID: "41a6c18f-f386-4739-8fa7-2acdfdf46223") : secret "cluster-monitoring-operator-tls" not found Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.759061 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.759075 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.759103 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.759096392 +0000 UTC m=+68.252925363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : secret "router-metrics-certs-default" not found Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:11:58.759158 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.759169 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle podName:eeaacab5-50b2-4ea4-8f6b-29eae71b790f nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.759155256 +0000 UTC m=+68.252984241 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle") pod "router-default-568fbddcbd-x8pzc" (UID: "eeaacab5-50b2-4ea4-8f6b-29eae71b790f") : configmap references non-existent config key: service-ca.crt Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.759200 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 12:11:58.759265 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:11:58.759224 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert podName:133dbdd2-a362-4d8c-adfd-fd1dca5ff65d nodeName:}" failed. No retries permitted until 2026-07-21 12:12:14.759217726 +0000 UTC m=+68.253046697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-22p7z" (UID: "133dbdd2-a362-4d8c-adfd-fd1dca5ff65d") : secret "networking-console-plugin-cert" not found Jul 21 12:12:02.598116 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:02.598081 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:12:02.598689 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:02.598173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:12:02.598689 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:02.598301 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:12:02.598689 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:02.598362 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls podName:1f1663b6-40b5-4515-b534-041eb295876f nodeName:}" failed. No retries permitted until 2026-07-21 12:12:10.59834868 +0000 UTC m=+64.092177651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4mjkh" (UID: "1f1663b6-40b5-4515-b534-041eb295876f") : secret "insights-runtime-extractor-tls" not found Jul 21 12:12:02.598689 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:02.598653 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f1663b6-40b5-4515-b534-041eb295876f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:12:03.190599 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:03.190568 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:12:03.190776 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:03.190620 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:12:03.190972 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:03.190960 2550 scope.go:117] "RemoveContainer" containerID="143480cd7a8597ca8e2a44acd20fa05eba4691420d0c1e91dabf015cfc4986d0" Jul 21 12:12:03.191120 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:03.191101 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-dvskp_openshift-console-operator(4fc860f4-873b-4e1e-8e3c-503f24ab6d86)\"" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" podUID="4fc860f4-873b-4e1e-8e3c-503f24ab6d86" Jul 21 12:12:09.279558 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:09.279528 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwhch" Jul 21 12:12:10.671418 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:10.671366 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:12:10.673680 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:10.673660 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f1663b6-40b5-4515-b534-041eb295876f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4mjkh\" (UID: \"1f1663b6-40b5-4515-b534-041eb295876f\") " pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:12:10.747564 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:10.747537 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2gbfg\"" Jul 21 12:12:10.755781 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:10.755763 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4mjkh" Jul 21 12:12:10.912994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:10.912872 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4mjkh"] Jul 21 12:12:10.915508 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:10.915472 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1663b6_40b5_4515_b534_041eb295876f.slice/crio-0e4c88ce1efafdad21eaa555fb95b66fd7929775932851804ff9d4c3ab6e4666 WatchSource:0}: Error finding container 0e4c88ce1efafdad21eaa555fb95b66fd7929775932851804ff9d4c3ab6e4666: Status 404 returned error can't find the container with id 0e4c88ce1efafdad21eaa555fb95b66fd7929775932851804ff9d4c3ab6e4666 Jul 21 12:12:11.446104 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.446068 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4mjkh" event={"ID":"1f1663b6-40b5-4515-b534-041eb295876f","Type":"ContainerStarted","Data":"0e4c88ce1efafdad21eaa555fb95b66fd7929775932851804ff9d4c3ab6e4666"} Jul 21 12:12:11.681786 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.681745 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:12:11.682264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.681803 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:12:11.684642 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.684586 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:12:11.684779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.684665 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:12:11.694806 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.694777 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4066c5a-25b5-4a23-80f7-ef95d3516d70-original-pull-secret\") pod \"global-pull-secret-syncer-rrb6l\" (UID: \"a4066c5a-25b5-4a23-80f7-ef95d3516d70\") " pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:12:11.694943 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.694836 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a385e43d-14a9-4f43-a962-a08a8523cb5e-metrics-certs\") pod \"network-metrics-daemon-bgmwr\" (UID: \"a385e43d-14a9-4f43-a962-a08a8523cb5e\") " pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:12:11.747771 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.747689 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2w9zn\"" Jul 21 12:12:11.752016 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.751763 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rrb6l" Jul 21 12:12:11.754693 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.754517 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bgmwr" Jul 21 12:12:11.782987 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.782902 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:12:11.786382 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.786358 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjjrl\" (UniqueName: \"kubernetes.io/projected/8888d463-620e-41f9-8f23-222532c774c0-kube-api-access-sjjrl\") pod \"network-check-target-jknc7\" (UID: \"8888d463-620e-41f9-8f23-222532c774c0\") " pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:12:11.932392 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.932315 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rrb6l"] Jul 21 12:12:11.937581 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:11.937540 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4066c5a_25b5_4a23_80f7_ef95d3516d70.slice/crio-32e077c75c1ff38591d3030a6230423ea571e7c9bcd51caf8fa8d2b746127669 WatchSource:0}: Error finding container 32e077c75c1ff38591d3030a6230423ea571e7c9bcd51caf8fa8d2b746127669: Status 404 returned error can't find the container with id 32e077c75c1ff38591d3030a6230423ea571e7c9bcd51caf8fa8d2b746127669 Jul 21 12:12:11.938848 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:11.938710 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bgmwr"] Jul 21 12:12:11.941519 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:11.941486 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda385e43d_14a9_4f43_a962_a08a8523cb5e.slice/crio-9e7c5b883dac935db4c57ea041e81e26a9c2388b7569555cc7535bf48d7f6951 WatchSource:0}: Error finding container 9e7c5b883dac935db4c57ea041e81e26a9c2388b7569555cc7535bf48d7f6951: Status 404 returned error can't find the container with id 9e7c5b883dac935db4c57ea041e81e26a9c2388b7569555cc7535bf48d7f6951 Jul 21 12:12:12.037484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.037401 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8v447\"" Jul 21 12:12:12.045640 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.045605 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:12:12.191603 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.191569 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jknc7"] Jul 21 12:12:12.194362 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:12.194336 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8888d463_620e_41f9_8f23_222532c774c0.slice/crio-d4f06dcb9d79d802923ec360e1ffa192316f85699e0e1eca910713d83efd4bc5 WatchSource:0}: Error finding container d4f06dcb9d79d802923ec360e1ffa192316f85699e0e1eca910713d83efd4bc5: Status 404 returned error can't find the container with id d4f06dcb9d79d802923ec360e1ffa192316f85699e0e1eca910713d83efd4bc5 Jul 21 12:12:12.451952 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.451913 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jknc7" event={"ID":"8888d463-620e-41f9-8f23-222532c774c0","Type":"ContainerStarted","Data":"b67e3a2c633eaa9b8e12b44ead5ed45f8785e2192fc0b0d8734fc7503b3c31b4"} Jul 21 12:12:12.452135 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.451974 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jknc7" event={"ID":"8888d463-620e-41f9-8f23-222532c774c0","Type":"ContainerStarted","Data":"d4f06dcb9d79d802923ec360e1ffa192316f85699e0e1eca910713d83efd4bc5"} Jul 21 12:12:12.452135 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.452021 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:12:12.453625 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.453564 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bgmwr" event={"ID":"a385e43d-14a9-4f43-a962-a08a8523cb5e","Type":"ContainerStarted","Data":"9e7c5b883dac935db4c57ea041e81e26a9c2388b7569555cc7535bf48d7f6951"} Jul 21 12:12:12.455001 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:12.454976 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rrb6l" event={"ID":"a4066c5a-25b5-4a23-80f7-ef95d3516d70","Type":"ContainerStarted","Data":"32e077c75c1ff38591d3030a6230423ea571e7c9bcd51caf8fa8d2b746127669"} Jul 21 12:12:14.711693 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.711656 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:12:14.712256 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.711928 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:12:14.714925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.714875 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/30ad9c8e-526a-4cbd-973d-8719f34ead2e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gqnh2\" (UID: \"30ad9c8e-526a-4cbd-973d-8719f34ead2e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:12:14.714925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.714909 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"image-registry-6c848ff744-k8fkq\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:12:14.813402 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.813351 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:12:14.814040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.813451 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:12:14.814040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.813502 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:12:14.814040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.813524 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:14.814040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.813575 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:14.814641 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.814591 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-service-ca-bundle\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:14.816411 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.816372 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a6c18f-f386-4739-8fa7-2acdfdf46223-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-7l28l\" (UID: \"41a6c18f-f386-4739-8fa7-2acdfdf46223\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:12:14.816540 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.816509 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/396d16ec-f45b-427e-846a-68ee756f846c-metrics-tls\") pod \"dns-default-nrn6d\" (UID: \"396d16ec-f45b-427e-846a-68ee756f846c\") " pod="openshift-dns/dns-default-nrn6d" Jul 21 12:12:14.816997 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.816974 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/133dbdd2-a362-4d8c-adfd-fd1dca5ff65d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-22p7z\" (UID: \"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:12:14.817691 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.817670 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eeaacab5-50b2-4ea4-8f6b-29eae71b790f-metrics-certs\") pod \"router-default-568fbddcbd-x8pzc\" (UID: \"eeaacab5-50b2-4ea4-8f6b-29eae71b790f\") " pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:14.922171 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.922139 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qpjlh\"" Jul 21 12:12:14.929463 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.929416 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" Jul 21 12:12:14.946524 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.946490 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l94b2\"" Jul 21 12:12:14.954710 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.954683 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:12:14.970201 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.970139 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r9cg8\"" Jul 21 12:12:14.978226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.978031 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" Jul 21 12:12:14.984800 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.984771 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4t9m4\"" Jul 21 12:12:14.992241 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.992216 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:14.999864 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:14.999842 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fnnf7\"" Jul 21 12:12:15.007439 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:15.007388 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ccdv6\"" Jul 21 12:12:15.007551 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:15.007464 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nrn6d" Jul 21 12:12:15.015029 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:15.015010 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" Jul 21 12:12:17.115738 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.115713 2550 scope.go:117] "RemoveContainer" containerID="143480cd7a8597ca8e2a44acd20fa05eba4691420d0c1e91dabf015cfc4986d0" Jul 21 12:12:17.628818 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.628634 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jknc7" podStartSLOduration=70.628608458 podStartE2EDuration="1m10.628608458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:12.471007837 +0000 UTC m=+65.964836828" watchObservedRunningTime="2026-07-21 12:12:17.628608458 +0000 UTC m=+71.122437452" Jul 21 12:12:17.629845 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.629666 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2"] Jul 21 12:12:17.706610 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.706583 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nrn6d"] Jul 21 12:12:17.789045 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:17.788817 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396d16ec_f45b_427e_846a_68ee756f846c.slice/crio-e567d00e7006483f7911a23a1a8ef09c02728aa28403b151cfdd315f236db123 WatchSource:0}: Error finding container e567d00e7006483f7911a23a1a8ef09c02728aa28403b151cfdd315f236db123: Status 404 returned error can't find the container with id e567d00e7006483f7911a23a1a8ef09c02728aa28403b151cfdd315f236db123 Jul 21 12:12:17.878829 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.878748 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z"] Jul 21 12:12:17.879712 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.879691 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l"] Jul 21 12:12:17.883220 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.883086 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c848ff744-k8fkq"] Jul 21 12:12:17.886103 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:17.886046 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133dbdd2_a362_4d8c_adfd_fd1dca5ff65d.slice/crio-2a43459576928a892bab38285e0514c43ddeebf861c17c019067d64d821d8b95 WatchSource:0}: Error finding container 2a43459576928a892bab38285e0514c43ddeebf861c17c019067d64d821d8b95: Status 404 returned error can't find the container with id 2a43459576928a892bab38285e0514c43ddeebf861c17c019067d64d821d8b95 Jul 21 12:12:17.915302 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:17.915276 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-568fbddcbd-x8pzc"] Jul 21 12:12:18.490529 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.490492 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4mjkh" event={"ID":"1f1663b6-40b5-4515-b534-041eb295876f","Type":"ContainerStarted","Data":"fc16e3b26605446ff009ba97349b1cf735e560889c08bea58392269ef25d92c8"} Jul 21 12:12:18.499143 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.499109 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rrb6l" event={"ID":"a4066c5a-25b5-4a23-80f7-ef95d3516d70","Type":"ContainerStarted","Data":"60cd692f022400a3c933b2a52659af221e9a47fb8d3b06ea5d3de0d79940f3df"} Jul 21 12:12:18.502090 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.502024 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" event={"ID":"41a6c18f-f386-4739-8fa7-2acdfdf46223","Type":"ContainerStarted","Data":"d1045ff33caab5e1330eb6dce167264b71ab8795a24317bb1edb1138bba906b0"} Jul 21 12:12:18.503884 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.503802 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nrn6d" event={"ID":"396d16ec-f45b-427e-846a-68ee756f846c","Type":"ContainerStarted","Data":"e567d00e7006483f7911a23a1a8ef09c02728aa28403b151cfdd315f236db123"} Jul 21 12:12:18.506646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.506623 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" event={"ID":"eeaacab5-50b2-4ea4-8f6b-29eae71b790f","Type":"ContainerStarted","Data":"9f192a2f166aca1858c2aa42a068e08b42618e12d5cd25ffbf922d54b97dabd9"} Jul 21 12:12:18.506745 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.506663 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" event={"ID":"eeaacab5-50b2-4ea4-8f6b-29eae71b790f","Type":"ContainerStarted","Data":"d5275ca2eb9514bd52b6023e694403917ea3be6b811f714bfc89cb27b8bdbe40"} Jul 21 12:12:18.507868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.507843 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" event={"ID":"30ad9c8e-526a-4cbd-973d-8719f34ead2e","Type":"ContainerStarted","Data":"a7a06bb7d7f1efd8ff2f8b4c70274b4969cf363c2ba9b8814d61cab77a885a0f"} Jul 21 12:12:18.510338 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.510274 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:12:18.510655 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.510351 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" event={"ID":"4fc860f4-873b-4e1e-8e3c-503f24ab6d86","Type":"ContainerStarted","Data":"d0fa7f27bcdd05dcf952d60ed112a0734164a5275f2a449b9416c999602df896"} Jul 21 12:12:18.513277 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.511175 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:12:18.518199 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.518092 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bgmwr" event={"ID":"a385e43d-14a9-4f43-a962-a08a8523cb5e","Type":"ContainerStarted","Data":"73f39fb22b396a6e9fee9c39686343a1d9f29876a82114264c4fafc7dce74483"} Jul 21 12:12:18.518199 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.518147 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bgmwr" event={"ID":"a385e43d-14a9-4f43-a962-a08a8523cb5e","Type":"ContainerStarted","Data":"568652b796fb52d47c38641093e8a77590aa4b24ab42cbd2c75112145e632b89"} Jul 21 12:12:18.518718 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.518696 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" Jul 21 12:12:18.520967 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.520945 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" event={"ID":"572604ec-8a1f-45fc-a6c6-26e661e4ff7d","Type":"ContainerStarted","Data":"adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4"} Jul 21 12:12:18.521129 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.521112 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" event={"ID":"572604ec-8a1f-45fc-a6c6-26e661e4ff7d","Type":"ContainerStarted","Data":"d30b5293c1786ca6a9350a0127c4c80a8a9993ea12203449f8b5dd85da505c36"} Jul 21 12:12:18.521363 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.521323 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:12:18.526584 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.526530 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" event={"ID":"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d","Type":"ContainerStarted","Data":"2a43459576928a892bab38285e0514c43ddeebf861c17c019067d64d821d8b95"} Jul 21 12:12:18.546751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.546053 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" podStartSLOduration=52.546026321 podStartE2EDuration="52.546026321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:18.544370759 +0000 UTC m=+72.038199753" watchObservedRunningTime="2026-07-21 12:12:18.546026321 +0000 UTC m=+72.039855366" Jul 21 12:12:18.546751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.546290 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rrb6l" podStartSLOduration=65.015442823 podStartE2EDuration="1m10.546278463s" totalImagesPullingTime="5.53083564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:18.517611295 +0000 UTC m=+72.011440294" watchObservedRunningTime="2026-07-21 12:12:18.546278463 +0000 UTC m=+72.040107457" Jul 21 12:12:18.569095 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.569032 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-dvskp" podStartSLOduration=43.123991531 podStartE2EDuration="52.569011453s" totalImagesPullingTime="9.445019922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:18.567797723 +0000 UTC m=+72.061626718" watchObservedRunningTime="2026-07-21 12:12:18.569011453 +0000 UTC m=+72.062840446" Jul 21 12:12:18.615999 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.615372 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" podStartSLOduration=70.615349912 podStartE2EDuration="1m10.615349912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:18.613846582 +0000 UTC m=+72.107675575" watchObservedRunningTime="2026-07-21 12:12:18.615349912 +0000 UTC m=+72.109178907" Jul 21 12:12:18.705168 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.704798 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bgmwr" podStartSLOduration=65.80406372 podStartE2EDuration="1m11.704780037s" totalImagesPullingTime="5.900716317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:18.643054227 +0000 UTC m=+72.136883223" watchObservedRunningTime="2026-07-21 12:12:18.704780037 +0000 UTC m=+72.198609031" Jul 21 12:12:18.705355 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.705323 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-thrrb"] Jul 21 12:12:18.708861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.708839 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:18.712119 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.711835 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-w67z7\"" Jul 21 12:12:18.712119 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.711917 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 12:12:18.712119 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.711835 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 12:12:18.720091 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.720035 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-thrrb"] Jul 21 12:12:18.852541 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.852454 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76g7\" (UniqueName: \"kubernetes.io/projected/f9b9baf9-31e6-42c8-b1fa-fa4f357396a0-kube-api-access-v76g7\") pod \"downloads-5bc67dcd46-thrrb\" (UID: \"f9b9baf9-31e6-42c8-b1fa-fa4f357396a0\") " pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:18.953798 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.953682 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76g7\" (UniqueName: \"kubernetes.io/projected/f9b9baf9-31e6-42c8-b1fa-fa4f357396a0-kube-api-access-v76g7\") pod \"downloads-5bc67dcd46-thrrb\" (UID: \"f9b9baf9-31e6-42c8-b1fa-fa4f357396a0\") " pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:18.968135 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.968061 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76g7\" (UniqueName: \"kubernetes.io/projected/f9b9baf9-31e6-42c8-b1fa-fa4f357396a0-kube-api-access-v76g7\") pod \"downloads-5bc67dcd46-thrrb\" (UID: \"f9b9baf9-31e6-42c8-b1fa-fa4f357396a0\") " pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:18.992751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.992721 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:18.995758 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:18.995730 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:19.021554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:19.021522 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:19.530639 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:19.530282 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:19.531575 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:19.531542 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-568fbddcbd-x8pzc" Jul 21 12:12:20.550254 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:20.550123 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-thrrb"] Jul 21 12:12:20.977082 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:20.977040 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b9baf9_31e6_42c8_b1fa_fa4f357396a0.slice/crio-6519809a6ec7aef346cfb1f61a0143e762bb1ff8984189d0c9b3df9ef298f3f7 WatchSource:0}: Error finding container 6519809a6ec7aef346cfb1f61a0143e762bb1ff8984189d0c9b3df9ef298f3f7: Status 404 returned error can't find the container with id 6519809a6ec7aef346cfb1f61a0143e762bb1ff8984189d0c9b3df9ef298f3f7 Jul 21 12:12:21.546159 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.546053 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" event={"ID":"133dbdd2-a362-4d8c-adfd-fd1dca5ff65d","Type":"ContainerStarted","Data":"aded6e6fbbe3a44c76c5849110c965891c28f407aca36704642ab9694a1ec186"} Jul 21 12:12:21.549020 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.548987 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" event={"ID":"41a6c18f-f386-4739-8fa7-2acdfdf46223","Type":"ContainerStarted","Data":"13e356f5de6defe6a0943c5cb6956d268afb5c21bc20a7dd0fd7affcc73688b9"} Jul 21 12:12:21.550533 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.550483 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-thrrb" event={"ID":"f9b9baf9-31e6-42c8-b1fa-fa4f357396a0","Type":"ContainerStarted","Data":"6519809a6ec7aef346cfb1f61a0143e762bb1ff8984189d0c9b3df9ef298f3f7"} Jul 21 12:12:21.553656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.553161 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nrn6d" event={"ID":"396d16ec-f45b-427e-846a-68ee756f846c","Type":"ContainerStarted","Data":"c020775aa3a390d5759b56cc74aef151e793ade1d93d9dfa83a37099fb9ed6e0"} Jul 21 12:12:21.553656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.553219 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nrn6d" event={"ID":"396d16ec-f45b-427e-846a-68ee756f846c","Type":"ContainerStarted","Data":"cbf5af56bbf95e80bf12daddee257f2b9839880de64371d7209c0acf9c07b83a"} Jul 21 12:12:21.553656 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.553251 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nrn6d" Jul 21 12:12:21.555678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.555525 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" event={"ID":"30ad9c8e-526a-4cbd-973d-8719f34ead2e","Type":"ContainerStarted","Data":"c48cbee6300c11a2e9099140fff6c4e6566daa4fc9f3b2f1296283462c593de1"} Jul 21 12:12:21.565604 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.565550 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-22p7z" podStartSLOduration=39.447578093 podStartE2EDuration="42.565535034s" totalImagesPullingTime="3.117956941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:21.564797902 +0000 UTC m=+75.058626896" watchObservedRunningTime="2026-07-21 12:12:21.565535034 +0000 UTC m=+75.059364027" Jul 21 12:12:21.611547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.611170 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-7l28l" podStartSLOduration=52.473333795 podStartE2EDuration="55.611146058s" totalImagesPullingTime="3.137812263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:21.586904371 +0000 UTC m=+75.080733366" watchObservedRunningTime="2026-07-21 12:12:21.611146058 +0000 UTC m=+75.104975055" Jul 21 12:12:21.611712 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:21.611554 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nrn6d" podStartSLOduration=36.38058438 podStartE2EDuration="39.611544656s" totalImagesPullingTime="3.230960276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:21.609727147 +0000 UTC m=+75.103556140" watchObservedRunningTime="2026-07-21 12:12:21.611544656 +0000 UTC m=+75.105373649" Jul 21 12:12:24.571643 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:24.571603 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" event={"ID":"30ad9c8e-526a-4cbd-973d-8719f34ead2e","Type":"ContainerStarted","Data":"e22c86e6a3ebe730085cef44c6fbda194ade17694ef578f7a32c9ec03ce09f69"} Jul 21 12:12:24.573564 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:24.573534 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4mjkh" event={"ID":"1f1663b6-40b5-4515-b534-041eb295876f","Type":"ContainerStarted","Data":"64152b059eeaca4350ad6235ddb72a7590c28c30a817b25a0ded8697e526ee4d"} Jul 21 12:12:24.590879 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:24.590826 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gqnh2" podStartSLOduration=55.397732376 podStartE2EDuration="58.590812247s" totalImagesPullingTime="3.193079871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:24.590343778 +0000 UTC m=+78.084172784" watchObservedRunningTime="2026-07-21 12:12:24.590812247 +0000 UTC m=+78.084641239" Jul 21 12:12:29.596470 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:29.596415 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4mjkh" event={"ID":"1f1663b6-40b5-4515-b534-041eb295876f","Type":"ContainerStarted","Data":"fbe9b189699605c08d305ada51c1459ee3db5dbd5b5c3a56569309bc3d810a15"} Jul 21 12:12:29.616075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:29.616023 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4mjkh" podStartSLOduration=17.929408944 podStartE2EDuration="35.616006949s" totalImagesPullingTime="17.686598005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:11:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:29.614577067 +0000 UTC m=+83.108406054" watchObservedRunningTime="2026-07-21 12:12:29.616006949 +0000 UTC m=+83.109835942" Jul 21 12:12:31.561946 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:31.561920 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nrn6d" Jul 21 12:12:38.107488 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.107438 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb"] Jul 21 12:12:38.270964 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.270901 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb"] Jul 21 12:12:38.271123 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.271022 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2j5wl"] Jul 21 12:12:38.271123 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.271033 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.273586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.273566 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 12:12:38.273762 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.273726 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 12:12:38.273912 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.273887 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 12:12:38.275162 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.274991 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-z8n2q\"" Jul 21 12:12:38.304779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.303757 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.309690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.308101 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 12:12:38.309690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.308341 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 12:12:38.309690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.308589 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 12:12:38.309690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.308768 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cj58p\"" Jul 21 12:12:38.313414 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.313351 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.313571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.313476 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljmd\" (UniqueName: \"kubernetes.io/projected/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-kube-api-access-rljmd\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.313571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.313531 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.313571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.313561 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.414375 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414291 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.414375 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414351 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414384 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-sys\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414408 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pttcl\" (UniqueName: \"kubernetes.io/projected/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-kube-api-access-pttcl\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414447 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-tls\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:38.414481 2550 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414508 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-root\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:38.414552 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-tls podName:2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7 nodeName:}" failed. No retries permitted until 2026-07-21 12:12:38.91452983 +0000 UTC m=+92.408358829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-dhlfb" (UID: "2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7") : secret "openshift-state-metrics-tls" not found Jul 21 12:12:38.414611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414575 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-textfile\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414616 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414658 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljmd\" (UniqueName: \"kubernetes.io/projected/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-kube-api-access-rljmd\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.414938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414712 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.414938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414746 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.414938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414784 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-wtmp\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.414938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.414816 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-metrics-client-ca\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.415552 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.415532 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.418460 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.417700 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.427385 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.427359 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljmd\" (UniqueName: \"kubernetes.io/projected/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-kube-api-access-rljmd\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.515615 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515576 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515611 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-sys\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515648 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pttcl\" (UniqueName: \"kubernetes.io/projected/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-kube-api-access-pttcl\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-tls\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515710 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-root\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515724 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-sys\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515736 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-textfile\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.515816 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515807 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516155 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515886 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-wtmp\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516155 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.515923 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-metrics-client-ca\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516155 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.516047 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-root\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516155 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:38.516066 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 12:12:38.516155 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:12:38.516124 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-tls podName:ec2db094-7e6c-4e30-a5fb-94cb23b33b8f nodeName:}" failed. No retries permitted until 2026-07-21 12:12:39.01610672 +0000 UTC m=+92.509935691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-tls") pod "node-exporter-2j5wl" (UID: "ec2db094-7e6c-4e30-a5fb-94cb23b33b8f") : secret "node-exporter-tls" not found Jul 21 12:12:38.516463 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.516165 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-wtmp\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516463 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.516049 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-textfile\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516463 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.516294 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.516674 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.516652 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-metrics-client-ca\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.518961 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.518937 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.526210 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.526188 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pttcl\" (UniqueName: \"kubernetes.io/projected/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-kube-api-access-pttcl\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:38.919663 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.919609 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:38.922464 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:38.922417 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-dhlfb\" (UID: \"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:39.020378 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:39.020342 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-tls\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:39.022867 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:39.022840 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec2db094-7e6c-4e30-a5fb-94cb23b33b8f-node-exporter-tls\") pod \"node-exporter-2j5wl\" (UID: \"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f\") " pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:39.182713 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:39.182679 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" Jul 21 12:12:39.219291 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:39.219265 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2j5wl" Jul 21 12:12:39.746308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:39.746275 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c848ff744-k8fkq"] Jul 21 12:12:39.751497 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:39.751463 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:12:40.078850 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:40.078813 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2db094_7e6c_4e30_a5fb_94cb23b33b8f.slice/crio-6d8d4b1adc55f307da9752eafd7b453a48754217765df2428a3ce42b2f0d0bfc WatchSource:0}: Error finding container 6d8d4b1adc55f307da9752eafd7b453a48754217765df2428a3ce42b2f0d0bfc: Status 404 returned error can't find the container with id 6d8d4b1adc55f307da9752eafd7b453a48754217765df2428a3ce42b2f0d0bfc Jul 21 12:12:40.185729 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.185699 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4"] Jul 21 12:12:40.201065 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.201029 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4"] Jul 21 12:12:40.201229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.201138 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.204090 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.203959 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb"] Jul 21 12:12:40.204333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204313 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qxmpz\"" Jul 21 12:12:40.204414 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204320 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2agclv44783ec\"" Jul 21 12:12:40.204414 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204322 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 21 12:12:40.204942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204693 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 21 12:12:40.204942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204762 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 21 12:12:40.204942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204775 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 21 12:12:40.204942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.204822 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 21 12:12:40.207983 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:40.207964 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0b72d9_c5f6_4edf_8b4d_baa9b03381e7.slice/crio-7c8b571e43aa09bea601334309b714a06c2d96164fcdbf7fa1b2b09523e34955 WatchSource:0}: Error finding container 7c8b571e43aa09bea601334309b714a06c2d96164fcdbf7fa1b2b09523e34955: Status 404 returned error can't find the container with id 7c8b571e43aa09bea601334309b714a06c2d96164fcdbf7fa1b2b09523e34955 Jul 21 12:12:40.333045 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.332974 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333045 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333028 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-tls\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333238 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333061 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-metrics-client-ca\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333238 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333088 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333238 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333177 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gwnz\" (UniqueName: \"kubernetes.io/projected/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-kube-api-access-5gwnz\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333390 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333251 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-grpc-tls\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333390 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333294 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.333390 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.333324 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434263 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434229 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434263 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434266 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434521 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434293 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434521 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434413 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-tls\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434521 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434484 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-metrics-client-ca\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434655 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434521 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434655 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434568 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gwnz\" (UniqueName: \"kubernetes.io/projected/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-kube-api-access-5gwnz\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.434655 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.434634 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-grpc-tls\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.435332 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.435278 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-metrics-client-ca\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.437192 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.437161 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-tls\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.437391 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.437371 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-grpc-tls\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.437860 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.437840 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.437939 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.437923 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.438035 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.438015 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.438070 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.438056 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.442548 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.442529 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gwnz\" (UniqueName: \"kubernetes.io/projected/f5b0ad4d-1dad-455c-a0bd-6ea6c9724838-kube-api-access-5gwnz\") pod \"thanos-querier-6f45cdfd54-dqxv4\" (UID: \"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838\") " pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.519160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.519134 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:40.643729 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.643588 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" event={"ID":"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7","Type":"ContainerStarted","Data":"b067ae2914645301d678749d1093eba26319f845d6575d380633d975e5dadf59"} Jul 21 12:12:40.643729 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.643663 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" event={"ID":"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7","Type":"ContainerStarted","Data":"d07523cfb0368c8a18d03e6fc1dd637639a3c1ab13880f2bf68066ea0554bf35"} Jul 21 12:12:40.643729 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.643692 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" event={"ID":"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7","Type":"ContainerStarted","Data":"7c8b571e43aa09bea601334309b714a06c2d96164fcdbf7fa1b2b09523e34955"} Jul 21 12:12:40.645452 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.645382 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5wl" event={"ID":"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f","Type":"ContainerStarted","Data":"6d8d4b1adc55f307da9752eafd7b453a48754217765df2428a3ce42b2f0d0bfc"} Jul 21 12:12:40.647238 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.647213 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-thrrb" event={"ID":"f9b9baf9-31e6-42c8-b1fa-fa4f357396a0","Type":"ContainerStarted","Data":"33f32593c5baa9c33851eb6d34a68c28171503b6f278c3793ba090874861a936"} Jul 21 12:12:40.647465 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.647446 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:40.648776 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.648751 2550 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-thrrb container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" start-of-body= Jul 21 12:12:40.648851 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.648805 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-thrrb" podUID="f9b9baf9-31e6-42c8-b1fa-fa4f357396a0" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" Jul 21 12:12:40.664774 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.664733 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-thrrb" podStartSLOduration=3.191014037 podStartE2EDuration="22.664718861s" totalImagesPullingTime="19.473704824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:40.663994062 +0000 UTC m=+94.157823057" watchObservedRunningTime="2026-07-21 12:12:40.664718861 +0000 UTC m=+94.158547853" Jul 21 12:12:40.685197 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:40.685174 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4"] Jul 21 12:12:40.687633 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:40.687605 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b0ad4d_1dad_455c_a0bd_6ea6c9724838.slice/crio-1c49a48135f95028d629d0bdaa7c93cde93b476a2918918279a627dc0730e3fc WatchSource:0}: Error finding container 1c49a48135f95028d629d0bdaa7c93cde93b476a2918918279a627dc0730e3fc: Status 404 returned error can't find the container with id 1c49a48135f95028d629d0bdaa7c93cde93b476a2918918279a627dc0730e3fc Jul 21 12:12:41.655058 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:41.655018 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"1c49a48135f95028d629d0bdaa7c93cde93b476a2918918279a627dc0730e3fc"} Jul 21 12:12:41.669108 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:41.668821 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-thrrb" Jul 21 12:12:42.508705 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.508670 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79dc965d9c-cvbd8"] Jul 21 12:12:42.529974 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.528914 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79dc965d9c-cvbd8"] Jul 21 12:12:42.529974 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.529034 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.533154 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.533125 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 21 12:12:42.533408 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.533385 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 21 12:12:42.533556 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.533543 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 21 12:12:42.533627 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.533608 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rfzq5\"" Jul 21 12:12:42.533686 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.533677 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4ha6f8qch53ed\"" Jul 21 12:12:42.533769 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.533134 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 21 12:12:42.657456 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657406 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6bc3193f-d43b-4f80-a687-d34552366a92-audit-log\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.657928 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657464 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6bc3193f-d43b-4f80-a687-d34552366a92-metrics-server-audit-profiles\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.657928 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657563 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-client-ca-bundle\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.657928 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657625 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-secret-metrics-server-tls\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.657928 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657786 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bc3193f-d43b-4f80-a687-d34552366a92-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.657928 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657847 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2bh4\" (UniqueName: \"kubernetes.io/projected/6bc3193f-d43b-4f80-a687-d34552366a92-kube-api-access-q2bh4\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.658243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.657934 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-secret-metrics-server-client-certs\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.758925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.758849 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-client-ca-bundle\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.758925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.758912 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-secret-metrics-server-tls\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.759145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.758958 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bc3193f-d43b-4f80-a687-d34552366a92-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.759145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.758999 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2bh4\" (UniqueName: \"kubernetes.io/projected/6bc3193f-d43b-4f80-a687-d34552366a92-kube-api-access-q2bh4\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.759145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.759061 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-secret-metrics-server-client-certs\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.759145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.759098 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6bc3193f-d43b-4f80-a687-d34552366a92-audit-log\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.759145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.759126 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6bc3193f-d43b-4f80-a687-d34552366a92-metrics-server-audit-profiles\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.760335 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.760307 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6bc3193f-d43b-4f80-a687-d34552366a92-metrics-server-audit-profiles\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.760498 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.760308 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bc3193f-d43b-4f80-a687-d34552366a92-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.760498 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.760463 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6bc3193f-d43b-4f80-a687-d34552366a92-audit-log\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.763462 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.763414 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-secret-metrics-server-tls\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.763772 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.763726 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-secret-metrics-server-client-certs\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.764493 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.764450 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bc3193f-d43b-4f80-a687-d34552366a92-client-ca-bundle\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.769815 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.769697 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2bh4\" (UniqueName: \"kubernetes.io/projected/6bc3193f-d43b-4f80-a687-d34552366a92-kube-api-access-q2bh4\") pod \"metrics-server-79dc965d9c-cvbd8\" (UID: \"6bc3193f-d43b-4f80-a687-d34552366a92\") " pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:42.843815 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:42.843733 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:12:43.332853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.332819 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b784ddd67-dhlph"] Jul 21 12:12:43.351699 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.351315 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b784ddd67-dhlph"] Jul 21 12:12:43.351699 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.351495 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.354569 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.354539 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 21 12:12:43.354734 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.354556 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gds2v\"" Jul 21 12:12:43.355018 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.354999 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 21 12:12:43.355233 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.355027 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 21 12:12:43.355233 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.355103 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 21 12:12:43.355366 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.355237 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 21 12:12:43.364626 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.364604 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 21 12:12:43.462538 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.462506 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jknc7" Jul 21 12:12:43.466244 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466222 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s84g7\" (UniqueName: \"kubernetes.io/projected/b50e76da-c961-48c0-bcfc-450e5a4bba97-kube-api-access-s84g7\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466364 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466268 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-serving-certs-ca-bundle\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466364 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466296 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466490 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466375 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-federate-client-tls\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466490 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466405 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-telemeter-client-tls\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466587 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466521 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466648 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466621 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-secret-telemeter-client\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.466696 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.466657 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-metrics-client-ca\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575025 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-federate-client-tls\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575078 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-telemeter-client-tls\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575148 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575233 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-secret-telemeter-client\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575270 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-metrics-client-ca\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575299 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s84g7\" (UniqueName: \"kubernetes.io/projected/b50e76da-c961-48c0-bcfc-450e5a4bba97-kube-api-access-s84g7\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575348 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-serving-certs-ca-bundle\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.575384 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.577229 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.578328 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.577924 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-metrics-client-ca\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.581972 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.581880 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.584368 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.582214 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b50e76da-c961-48c0-bcfc-450e5a4bba97-serving-certs-ca-bundle\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.584368 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.583986 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-secret-telemeter-client\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.584368 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.584254 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-telemeter-client-tls\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.584622 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.584575 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b50e76da-c961-48c0-bcfc-450e5a4bba97-federate-client-tls\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.594067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.593986 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s84g7\" (UniqueName: \"kubernetes.io/projected/b50e76da-c961-48c0-bcfc-450e5a4bba97-kube-api-access-s84g7\") pod \"telemeter-client-7b784ddd67-dhlph\" (UID: \"b50e76da-c961-48c0-bcfc-450e5a4bba97\") " pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.620359 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.620214 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79dc965d9c-cvbd8"] Jul 21 12:12:43.622977 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:43.622934 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc3193f_d43b_4f80_a687_d34552366a92.slice/crio-0d6a16f6baa61fdba7c6232ea596ec078c36d64fc9c7e1e138e2f0df94f3ded8 WatchSource:0}: Error finding container 0d6a16f6baa61fdba7c6232ea596ec078c36d64fc9c7e1e138e2f0df94f3ded8: Status 404 returned error can't find the container with id 0d6a16f6baa61fdba7c6232ea596ec078c36d64fc9c7e1e138e2f0df94f3ded8 Jul 21 12:12:43.670646 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.670250 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" event={"ID":"2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7","Type":"ContainerStarted","Data":"25aeb94589bbad2e64b3bd201e3d8f05d8173590959d1bb04d28c208918be154"} Jul 21 12:12:43.671078 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.670618 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" Jul 21 12:12:43.672620 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.672558 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5wl" event={"ID":"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f","Type":"ContainerStarted","Data":"f9b728b6a58678df3a03cc32a6342bff81c149b58a0b29078bd0af06274ef802"} Jul 21 12:12:43.674193 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.674165 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" event={"ID":"6bc3193f-d43b-4f80-a687-d34552366a92","Type":"ContainerStarted","Data":"0d6a16f6baa61fdba7c6232ea596ec078c36d64fc9c7e1e138e2f0df94f3ded8"} Jul 21 12:12:43.691167 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.691108 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-dhlfb" podStartSLOduration=2.832918149 podStartE2EDuration="5.691088564s" totalImagesPullingTime="2.858170415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:43.689648097 +0000 UTC m=+97.183477096" watchObservedRunningTime="2026-07-21 12:12:43.691088564 +0000 UTC m=+97.184917558" Jul 21 12:12:43.966524 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:43.966493 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b784ddd67-dhlph"] Jul 21 12:12:43.971397 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:43.971372 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50e76da_c961_48c0_bcfc_450e5a4bba97.slice/crio-0ad135210d3146e1c149589a4e76ea1b827c6ea4b2a3d6f36bfa09ed926b6e43 WatchSource:0}: Error finding container 0ad135210d3146e1c149589a4e76ea1b827c6ea4b2a3d6f36bfa09ed926b6e43: Status 404 returned error can't find the container with id 0ad135210d3146e1c149589a4e76ea1b827c6ea4b2a3d6f36bfa09ed926b6e43 Jul 21 12:12:44.684482 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:44.684373 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" event={"ID":"b50e76da-c961-48c0-bcfc-450e5a4bba97","Type":"ContainerStarted","Data":"0ad135210d3146e1c149589a4e76ea1b827c6ea4b2a3d6f36bfa09ed926b6e43"} Jul 21 12:12:44.687965 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:44.686978 2550 generic.go:356] "Generic (PLEG): container finished" podID="ec2db094-7e6c-4e30-a5fb-94cb23b33b8f" containerID="f9b728b6a58678df3a03cc32a6342bff81c149b58a0b29078bd0af06274ef802" exitCode=0 Jul 21 12:12:44.687965 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:44.687065 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5wl" event={"ID":"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f","Type":"ContainerDied","Data":"f9b728b6a58678df3a03cc32a6342bff81c149b58a0b29078bd0af06274ef802"} Jul 21 12:12:44.694064 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:44.694035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"68de0d1ee57df3f2f9a995166c2fe5a6e68f85ef85c851b998ce2fb7782c9103"} Jul 21 12:12:44.694411 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:44.694104 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"0e8ff3915ec7734758928225da327d66c52ee7094e52e5dd808d416d5a93db5d"} Jul 21 12:12:44.694411 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:44.694152 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"d5e24415221abf83c18f474eab7c2efffd979a615ffb82806548ec758497d26c"} Jul 21 12:12:45.623089 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.623048 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75879fd8f6-88j77"] Jul 21 12:12:45.646639 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.646607 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75879fd8f6-88j77"] Jul 21 12:12:45.646880 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.646727 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.649630 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.649581 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 12:12:45.649758 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.649703 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kkd5b\"" Jul 21 12:12:45.649758 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.649701 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 12:12:45.650805 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.650696 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 12:12:45.650941 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.650912 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 12:12:45.651479 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.651460 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 12:12:45.657993 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.657923 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 12:12:45.702253 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.702218 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5wl" event={"ID":"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f","Type":"ContainerStarted","Data":"cf11cfd75ec5c4f053510171240f6bb73445bd0273f2f21206213117287ae418"} Jul 21 12:12:45.702690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.702269 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5wl" event={"ID":"ec2db094-7e6c-4e30-a5fb-94cb23b33b8f","Type":"ContainerStarted","Data":"3096b7ae015cb1bccca4abbacffc801d23cfeb57677091128fcad99acba3992a"} Jul 21 12:12:45.722436 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.722386 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2j5wl" podStartSLOduration=4.258992831 podStartE2EDuration="7.722369811s" totalImagesPullingTime="3.366550787s" totalInitContainerRuntime="96.826193ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:45.721543598 +0000 UTC m=+99.215372592" watchObservedRunningTime="2026-07-21 12:12:45.722369811 +0000 UTC m=+99.216198803" Jul 21 12:12:45.801762 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.801725 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-serving-cert\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.801933 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.801790 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hdh2\" (UniqueName: \"kubernetes.io/projected/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-kube-api-access-6hdh2\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.801933 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.801832 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-service-ca\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.802062 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.801944 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-config\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.802062 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.802005 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-trusted-ca-bundle\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.802062 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.802041 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-oauth-config\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.802221 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.802071 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-oauth-serving-cert\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903449 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-trusted-ca-bundle\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903521 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-oauth-config\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903549 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-oauth-serving-cert\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903619 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-serving-cert\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903655 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hdh2\" (UniqueName: \"kubernetes.io/projected/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-kube-api-access-6hdh2\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903704 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-service-ca\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.903889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.903749 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-config\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.904595 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.904406 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-trusted-ca-bundle\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.904595 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.904523 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-oauth-serving-cert\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.904806 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.904784 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-service-ca\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.905192 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.905170 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-config\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.907174 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.907152 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-serving-cert\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.907297 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.907240 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-oauth-config\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.912827 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.912801 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hdh2\" (UniqueName: \"kubernetes.io/projected/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-kube-api-access-6hdh2\") pod \"console-75879fd8f6-88j77\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:45.966125 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:45.966091 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:12:47.321753 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.321446 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75879fd8f6-88j77"] Jul 21 12:12:47.325903 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:12:47.325846 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13ac515_de97_4fcd_8ab3_9b0a4faa913f.slice/crio-5d8b6ba9e4de19c70bf537a611a12d77f7bd9439df45359c8f915894115bee27 WatchSource:0}: Error finding container 5d8b6ba9e4de19c70bf537a611a12d77f7bd9439df45359c8f915894115bee27: Status 404 returned error can't find the container with id 5d8b6ba9e4de19c70bf537a611a12d77f7bd9439df45359c8f915894115bee27 Jul 21 12:12:47.714161 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.714075 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75879fd8f6-88j77" event={"ID":"f13ac515-de97-4fcd-8ab3-9b0a4faa913f","Type":"ContainerStarted","Data":"5d8b6ba9e4de19c70bf537a611a12d77f7bd9439df45359c8f915894115bee27"} Jul 21 12:12:47.716243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.716207 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" event={"ID":"6bc3193f-d43b-4f80-a687-d34552366a92","Type":"ContainerStarted","Data":"e49ba8e0dcbe9e0b6875228d0134139d1891e84cb4a6ba0e30dcb79f3b4451e6"} Jul 21 12:12:47.720025 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.719993 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"5090fba0ceb5a7f1637b285e307a6323141215d37218b8b88ec4f52c7bdcbe9c"} Jul 21 12:12:47.720159 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.720062 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"60eb6da189794c3afb8758580d3589918fd989ef254bc32eefc3148ed5a64a53"} Jul 21 12:12:47.720159 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.720105 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" event={"ID":"f5b0ad4d-1dad-455c-a0bd-6ea6c9724838","Type":"ContainerStarted","Data":"9dc20bb4e7730cc9bf844a303c27e8aaeccc1cf3f87c0e2646162d139a0f73ec"} Jul 21 12:12:47.720159 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.720150 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:47.721718 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.721691 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" event={"ID":"b50e76da-c961-48c0-bcfc-450e5a4bba97","Type":"ContainerStarted","Data":"26de1a68f7d48d9da53e70b52a46c522a361817a6d5adfdb7ed7da7af322b486"} Jul 21 12:12:47.736271 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.735208 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" podStartSLOduration=2.21761632 podStartE2EDuration="5.735186807s" totalImagesPullingTime="3.517570487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:47.734467149 +0000 UTC m=+101.228296144" watchObservedRunningTime="2026-07-21 12:12:47.735186807 +0000 UTC m=+101.229015801" Jul 21 12:12:47.756638 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:47.756578 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" podStartSLOduration=1.696307815 podStartE2EDuration="7.756557993s" totalImagesPullingTime="6.060250178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:47.755708953 +0000 UTC m=+101.249537959" watchObservedRunningTime="2026-07-21 12:12:47.756557993 +0000 UTC m=+101.250386988" Jul 21 12:12:49.737238 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:49.737146 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" event={"ID":"b50e76da-c961-48c0-bcfc-450e5a4bba97","Type":"ContainerStarted","Data":"1bb0e4a7d4873d3f57c0d650460a614645a647b690670bf81f4eac78124149ff"} Jul 21 12:12:49.737238 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:49.737216 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" event={"ID":"b50e76da-c961-48c0-bcfc-450e5a4bba97","Type":"ContainerStarted","Data":"0260413c28c655328e2fb7a94030fc0aacf4984cea47aeeebbc89c8ca9f990c9"} Jul 21 12:12:49.766018 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:49.765956 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b784ddd67-dhlph" podStartSLOduration=1.746664558 podStartE2EDuration="6.765937613s" totalImagesPullingTime="5.019273055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:49.762256732 +0000 UTC m=+103.256085724" watchObservedRunningTime="2026-07-21 12:12:49.765937613 +0000 UTC m=+103.259766632" Jul 21 12:12:50.678436 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:50.678382 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75879fd8f6-88j77"] Jul 21 12:12:51.752685 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:51.752650 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75879fd8f6-88j77" event={"ID":"f13ac515-de97-4fcd-8ab3-9b0a4faa913f","Type":"ContainerStarted","Data":"504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f"} Jul 21 12:12:51.770952 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:51.770893 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75879fd8f6-88j77" podStartSLOduration=2.784144896 podStartE2EDuration="6.770871029s" totalImagesPullingTime="3.986726133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:12:51.770585276 +0000 UTC m=+105.264414269" watchObservedRunningTime="2026-07-21 12:12:51.770871029 +0000 UTC m=+105.264700024" Jul 21 12:12:53.735797 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:53.735765 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6f45cdfd54-dqxv4" Jul 21 12:12:55.966396 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:12:55.966365 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:13:02.844702 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:02.844670 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:13:02.845162 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:02.844720 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:13:04.769332 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:04.769283 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" podUID="572604ec-8a1f-45fc-a6c6-26e661e4ff7d" containerName="registry" containerID="cri-o://adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4" gracePeriod=30 Jul 21 12:13:05.043704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.043627 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:13:05.092520 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092489 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-image-registry-private-configuration\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092549 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092573 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-trusted-ca\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092598 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-bound-sa-token\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092637 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khxh8\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-kube-api-access-khxh8\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092700 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-ca-trust-extracted\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092958 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092727 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-certificates\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.092958 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.092757 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-installation-pull-secrets\") pod \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\" (UID: \"572604ec-8a1f-45fc-a6c6-26e661e4ff7d\") " Jul 21 12:13:05.093209 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.093152 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:13:05.093353 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.093265 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:13:05.095295 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.095249 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:13:05.095452 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.095409 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:13:05.095782 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.095691 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:13:05.095782 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.095752 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-kube-api-access-khxh8" (OuterVolumeSpecName: "kube-api-access-khxh8") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "kube-api-access-khxh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:13:05.095914 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.095783 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:13:05.101812 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.101788 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "572604ec-8a1f-45fc-a6c6-26e661e4ff7d" (UID: "572604ec-8a1f-45fc-a6c6-26e661e4ff7d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:13:05.193841 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193808 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-image-registry-private-configuration\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.193841 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193839 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-tls\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.193841 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193849 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-trusted-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.194081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193858 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-bound-sa-token\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.194081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193866 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khxh8\" (UniqueName: \"kubernetes.io/projected/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-kube-api-access-khxh8\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.194081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193875 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-ca-trust-extracted\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.194081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193883 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-registry-certificates\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.194081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.193891 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572604ec-8a1f-45fc-a6c6-26e661e4ff7d-installation-pull-secrets\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:05.812285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.812253 2550 generic.go:356] "Generic (PLEG): container finished" podID="572604ec-8a1f-45fc-a6c6-26e661e4ff7d" containerID="adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4" exitCode=0 Jul 21 12:13:05.812737 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.812296 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" event={"ID":"572604ec-8a1f-45fc-a6c6-26e661e4ff7d","Type":"ContainerDied","Data":"adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4"} Jul 21 12:13:05.812737 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.812302 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" Jul 21 12:13:05.812737 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.812337 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c848ff744-k8fkq" event={"ID":"572604ec-8a1f-45fc-a6c6-26e661e4ff7d","Type":"ContainerDied","Data":"d30b5293c1786ca6a9350a0127c4c80a8a9993ea12203449f8b5dd85da505c36"} Jul 21 12:13:05.812737 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.812355 2550 scope.go:117] "RemoveContainer" containerID="adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4" Jul 21 12:13:05.820780 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.820762 2550 scope.go:117] "RemoveContainer" containerID="adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4" Jul 21 12:13:05.821065 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:13:05.821047 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4\": container with ID starting with adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4 not found: ID does not exist" containerID="adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4" Jul 21 12:13:05.821118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.821071 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4"} err="failed to get container status \"adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4\": rpc error: code = NotFound desc = could not find container \"adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4\": container with ID starting with adbfe6fd28a6fa28bfdc341c56283b40242b1b95088895a153365d98a6e7f9d4 not found: ID does not exist" Jul 21 12:13:05.829873 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.829844 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c848ff744-k8fkq"] Jul 21 12:13:05.833491 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:05.833467 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c848ff744-k8fkq"] Jul 21 12:13:07.119289 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:07.119256 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572604ec-8a1f-45fc-a6c6-26e661e4ff7d" path="/var/lib/kubelet/pods/572604ec-8a1f-45fc-a6c6-26e661e4ff7d/volumes" Jul 21 12:13:09.829967 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:09.829934 2550 generic.go:356] "Generic (PLEG): container finished" podID="a380abd7-3ed2-4c49-b8b7-18431a1f7469" containerID="182407b0e277ea923e0e5f3ff62237fa001196b49141562ee52eae8e2e519ca6" exitCode=0 Jul 21 12:13:09.830395 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:09.830002 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-zsqww" event={"ID":"a380abd7-3ed2-4c49-b8b7-18431a1f7469","Type":"ContainerDied","Data":"182407b0e277ea923e0e5f3ff62237fa001196b49141562ee52eae8e2e519ca6"} Jul 21 12:13:09.830395 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:09.830384 2550 scope.go:117] "RemoveContainer" containerID="182407b0e277ea923e0e5f3ff62237fa001196b49141562ee52eae8e2e519ca6" Jul 21 12:13:10.610005 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:10.609978 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568fbddcbd-x8pzc_eeaacab5-50b2-4ea4-8f6b-29eae71b790f/router/0.log" Jul 21 12:13:10.616505 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:10.616486 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fl9m4_317bf622-f052-40df-be20-3269ccdf1cda/serve-healthcheck-canary/0.log" Jul 21 12:13:10.835885 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:10.835854 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-zsqww" event={"ID":"a380abd7-3ed2-4c49-b8b7-18431a1f7469","Type":"ContainerStarted","Data":"b7ca376f5739fa91d07820edb8be9668ec323054ae549acf99b429c259d6ea9f"} Jul 21 12:13:13.848878 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:13.848850 2550 generic.go:356] "Generic (PLEG): container finished" podID="05aaecb6-6e63-4345-b3a3-4a930d3ef9bd" containerID="df7c7daae8904dff41585dd9f58ec5d28f65e1c6d696e60b33ef60de84c4eca0" exitCode=0 Jul 21 12:13:13.849281 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:13.848903 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" event={"ID":"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd","Type":"ContainerDied","Data":"df7c7daae8904dff41585dd9f58ec5d28f65e1c6d696e60b33ef60de84c4eca0"} Jul 21 12:13:13.849281 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:13.849194 2550 scope.go:117] "RemoveContainer" containerID="df7c7daae8904dff41585dd9f58ec5d28f65e1c6d696e60b33ef60de84c4eca0" Jul 21 12:13:14.855337 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:14.855305 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-7hhmp" event={"ID":"05aaecb6-6e63-4345-b3a3-4a930d3ef9bd","Type":"ContainerStarted","Data":"75ce61ca5c1c5929e447150cc7026c5856c32be0fd000b2821558922013ddbe5"} Jul 21 12:13:16.773223 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:16.773176 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75879fd8f6-88j77" podUID="f13ac515-de97-4fcd-8ab3-9b0a4faa913f" containerName="console" containerID="cri-o://504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f" gracePeriod=15 Jul 21 12:13:17.085338 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.085312 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:13:17.195173 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195143 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-service-ca\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195173 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195184 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-config\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195417 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195216 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-serving-cert\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195417 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195251 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-oauth-serving-cert\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195417 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195270 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hdh2\" (UniqueName: \"kubernetes.io/projected/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-kube-api-access-6hdh2\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195417 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195285 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-oauth-config\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195417 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195338 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-trusted-ca-bundle\") pod \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\" (UID: \"f13ac515-de97-4fcd-8ab3-9b0a4faa913f\") " Jul 21 12:13:17.195690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195599 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-service-ca" (OuterVolumeSpecName: "service-ca") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:13:17.195811 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195747 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-config" (OuterVolumeSpecName: "console-config") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:13:17.195932 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195906 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:13:17.196049 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.195963 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:13:17.197778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.197746 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:13:17.197778 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.197761 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:13:17.197896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.197832 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-kube-api-access-6hdh2" (OuterVolumeSpecName: "kube-api-access-6hdh2") pod "f13ac515-de97-4fcd-8ab3-9b0a4faa913f" (UID: "f13ac515-de97-4fcd-8ab3-9b0a4faa913f"). InnerVolumeSpecName "kube-api-access-6hdh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:13:17.296383 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296353 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-trusted-ca-bundle\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.296383 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296378 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.296383 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296388 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-config\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.296611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296396 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-serving-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.296611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296405 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-oauth-serving-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.296611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296414 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hdh2\" (UniqueName: \"kubernetes.io/projected/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-kube-api-access-6hdh2\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.296611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.296437 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f13ac515-de97-4fcd-8ab3-9b0a4faa913f-console-oauth-config\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:13:17.868272 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.868239 2550 generic.go:356] "Generic (PLEG): container finished" podID="f13ac515-de97-4fcd-8ab3-9b0a4faa913f" containerID="504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f" exitCode=0 Jul 21 12:13:17.868809 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.868295 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75879fd8f6-88j77" Jul 21 12:13:17.868809 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.868309 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75879fd8f6-88j77" event={"ID":"f13ac515-de97-4fcd-8ab3-9b0a4faa913f","Type":"ContainerDied","Data":"504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f"} Jul 21 12:13:17.868809 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.868349 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75879fd8f6-88j77" event={"ID":"f13ac515-de97-4fcd-8ab3-9b0a4faa913f","Type":"ContainerDied","Data":"5d8b6ba9e4de19c70bf537a611a12d77f7bd9439df45359c8f915894115bee27"} Jul 21 12:13:17.868809 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.868368 2550 scope.go:117] "RemoveContainer" containerID="504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f" Jul 21 12:13:17.879860 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.879838 2550 scope.go:117] "RemoveContainer" containerID="504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f" Jul 21 12:13:17.880141 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:13:17.880114 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f\": container with ID starting with 504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f not found: ID does not exist" containerID="504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f" Jul 21 12:13:17.880187 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.880142 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f"} err="failed to get container status \"504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f\": rpc error: code = NotFound desc = could not find container \"504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f\": container with ID starting with 504fa86e99e757dc2ce05c330ed2f9ad7515ae7c652160c6bdc9431918ab781f not found: ID does not exist" Jul 21 12:13:17.892949 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.892923 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75879fd8f6-88j77"] Jul 21 12:13:17.896285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:17.896250 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75879fd8f6-88j77"] Jul 21 12:13:19.119421 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:19.119392 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13ac515-de97-4fcd-8ab3-9b0a4faa913f" path="/var/lib/kubelet/pods/f13ac515-de97-4fcd-8ab3-9b0a4faa913f/volumes" Jul 21 12:13:19.878475 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:19.878418 2550 generic.go:356] "Generic (PLEG): container finished" podID="7135034a-0b9e-44b6-9619-93b88cf3a501" containerID="4c2fe14892f51e6689d8e74b1856f245c6f371144c0875e66c8e84543f15e69f" exitCode=0 Jul 21 12:13:19.878650 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:19.878491 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" event={"ID":"7135034a-0b9e-44b6-9619-93b88cf3a501","Type":"ContainerDied","Data":"4c2fe14892f51e6689d8e74b1856f245c6f371144c0875e66c8e84543f15e69f"} Jul 21 12:13:19.878822 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:19.878805 2550 scope.go:117] "RemoveContainer" containerID="4c2fe14892f51e6689d8e74b1856f245c6f371144c0875e66c8e84543f15e69f" Jul 21 12:13:20.884129 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:20.884088 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-rgkhb" event={"ID":"7135034a-0b9e-44b6-9619-93b88cf3a501","Type":"ContainerStarted","Data":"51d1df46d33b64e4e5316c240a8e9b4aa5bafd6f0f71444614cb0c11bd4595b4"} Jul 21 12:13:22.850236 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:22.850205 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:13:22.854249 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:13:22.854225 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79dc965d9c-cvbd8" Jul 21 12:14:18.700799 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.700742 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cf7b8cdd6-swqnk"] Jul 21 12:14:18.701248 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.701211 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572604ec-8a1f-45fc-a6c6-26e661e4ff7d" containerName="registry" Jul 21 12:14:18.701248 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.701231 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="572604ec-8a1f-45fc-a6c6-26e661e4ff7d" containerName="registry" Jul 21 12:14:18.701248 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.701244 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13ac515-de97-4fcd-8ab3-9b0a4faa913f" containerName="console" Jul 21 12:14:18.701345 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.701253 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13ac515-de97-4fcd-8ab3-9b0a4faa913f" containerName="console" Jul 21 12:14:18.701401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.701386 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="572604ec-8a1f-45fc-a6c6-26e661e4ff7d" containerName="registry" Jul 21 12:14:18.701482 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.701403 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13ac515-de97-4fcd-8ab3-9b0a4faa913f" containerName="console" Jul 21 12:14:18.705957 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.705936 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.708647 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.708628 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 12:14:18.709840 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.709822 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 12:14:18.709957 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.709852 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 12:14:18.709957 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.709896 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kkd5b\"" Jul 21 12:14:18.710349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.710216 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 12:14:18.710349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.710297 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 12:14:18.714905 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.714883 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf7b8cdd6-swqnk"] Jul 21 12:14:18.715645 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.715629 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 12:14:18.813990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.813951 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-trusted-ca-bundle\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.814179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.814012 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-service-ca\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.814179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.814050 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-serving-cert\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.814179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.814071 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ldd\" (UniqueName: \"kubernetes.io/projected/1bfd305a-a4cd-4002-a389-249430b87ab1-kube-api-access-m5ldd\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.814179 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.814148 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-oauth-config\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.814325 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.814189 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-console-config\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.814325 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.814219 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-oauth-serving-cert\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915392 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915357 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-oauth-serving-cert\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915405 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-trusted-ca-bundle\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915465 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-service-ca\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915484 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-serving-cert\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915504 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ldd\" (UniqueName: \"kubernetes.io/projected/1bfd305a-a4cd-4002-a389-249430b87ab1-kube-api-access-m5ldd\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915537 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-oauth-config\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.915601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.915562 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-console-config\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.916210 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.916177 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-oauth-serving-cert\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.916406 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.916373 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-console-config\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.916583 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.916564 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-trusted-ca-bundle\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.916668 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.916644 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-service-ca\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.918350 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.918330 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-oauth-config\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.918467 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.918420 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-serving-cert\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:18.924304 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:18.924279 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ldd\" (UniqueName: \"kubernetes.io/projected/1bfd305a-a4cd-4002-a389-249430b87ab1-kube-api-access-m5ldd\") pod \"console-7cf7b8cdd6-swqnk\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:19.016920 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:19.016835 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:19.149032 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:19.149010 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf7b8cdd6-swqnk"] Jul 21 12:14:19.151141 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:14:19.151117 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfd305a_a4cd_4002_a389_249430b87ab1.slice/crio-8bc4687bdf567178291cc0ed8fce7d69a7477cbed94aa3d439666d2cc8dd72ce WatchSource:0}: Error finding container 8bc4687bdf567178291cc0ed8fce7d69a7477cbed94aa3d439666d2cc8dd72ce: Status 404 returned error can't find the container with id 8bc4687bdf567178291cc0ed8fce7d69a7477cbed94aa3d439666d2cc8dd72ce Jul 21 12:14:20.128366 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:20.128334 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf7b8cdd6-swqnk" event={"ID":"1bfd305a-a4cd-4002-a389-249430b87ab1","Type":"ContainerStarted","Data":"e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5"} Jul 21 12:14:20.128748 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:20.128385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf7b8cdd6-swqnk" event={"ID":"1bfd305a-a4cd-4002-a389-249430b87ab1","Type":"ContainerStarted","Data":"8bc4687bdf567178291cc0ed8fce7d69a7477cbed94aa3d439666d2cc8dd72ce"} Jul 21 12:14:29.017489 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:29.017417 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:29.017908 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:29.017511 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:29.022533 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:29.022509 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:14:29.043917 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:29.043869 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cf7b8cdd6-swqnk" podStartSLOduration=11.043852929 podStartE2EDuration="11.043852929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:14:20.146613474 +0000 UTC m=+193.640442468" watchObservedRunningTime="2026-07-21 12:14:29.043852929 +0000 UTC m=+202.537681991" Jul 21 12:14:29.168385 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:14:29.168356 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:16:06.921722 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:16:06.921693 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:16:06.922219 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:16:06.921963 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:16:06.931903 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:16:06.931880 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:16:06.932084 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:16:06.932068 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:16:06.935295 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:16:06.935269 2550 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 12:20:59.322332 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.322302 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tgpqz"] Jul 21 12:20:59.325519 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.325502 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.328567 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.328547 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ktr8m\"" Jul 21 12:20:59.329611 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.329586 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 12:20:59.329715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.329614 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 12:20:59.344459 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.344420 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qwdp\" (UniqueName: \"kubernetes.io/projected/68a5a155-98c8-46f0-96a9-54e04ff50dee-kube-api-access-6qwdp\") pod \"cert-manager-webhook-6486ffd997-tgpqz\" (UID: \"68a5a155-98c8-46f0-96a9-54e04ff50dee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.344601 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.344558 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68a5a155-98c8-46f0-96a9-54e04ff50dee-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tgpqz\" (UID: \"68a5a155-98c8-46f0-96a9-54e04ff50dee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.345069 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.345049 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tgpqz"] Jul 21 12:20:59.445304 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.445265 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qwdp\" (UniqueName: \"kubernetes.io/projected/68a5a155-98c8-46f0-96a9-54e04ff50dee-kube-api-access-6qwdp\") pod \"cert-manager-webhook-6486ffd997-tgpqz\" (UID: \"68a5a155-98c8-46f0-96a9-54e04ff50dee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.445511 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.445416 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68a5a155-98c8-46f0-96a9-54e04ff50dee-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tgpqz\" (UID: \"68a5a155-98c8-46f0-96a9-54e04ff50dee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.467355 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.467310 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68a5a155-98c8-46f0-96a9-54e04ff50dee-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tgpqz\" (UID: \"68a5a155-98c8-46f0-96a9-54e04ff50dee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.467896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.467874 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qwdp\" (UniqueName: \"kubernetes.io/projected/68a5a155-98c8-46f0-96a9-54e04ff50dee-kube-api-access-6qwdp\") pod \"cert-manager-webhook-6486ffd997-tgpqz\" (UID: \"68a5a155-98c8-46f0-96a9-54e04ff50dee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.652620 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.652524 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:20:59.787311 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.787226 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tgpqz"] Jul 21 12:20:59.790317 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:20:59.790290 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68a5a155_98c8_46f0_96a9_54e04ff50dee.slice/crio-667de9636440167080a0ed47bc231f59056a067d1978cb4185e3f585e2bbf6f4 WatchSource:0}: Error finding container 667de9636440167080a0ed47bc231f59056a067d1978cb4185e3f585e2bbf6f4: Status 404 returned error can't find the container with id 667de9636440167080a0ed47bc231f59056a067d1978cb4185e3f585e2bbf6f4 Jul 21 12:20:59.792255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:20:59.792239 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:21:00.755544 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:00.755508 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" event={"ID":"68a5a155-98c8-46f0-96a9-54e04ff50dee","Type":"ContainerStarted","Data":"667de9636440167080a0ed47bc231f59056a067d1978cb4185e3f585e2bbf6f4"} Jul 21 12:21:03.771034 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:03.770950 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" event={"ID":"68a5a155-98c8-46f0-96a9-54e04ff50dee","Type":"ContainerStarted","Data":"8707a25b489613a16d2f356e50c301c3aea7a807d3549c8fe65dbf3f54780d7a"} Jul 21 12:21:03.771034 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:03.771019 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:21:03.792788 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:03.792739 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" podStartSLOduration=1.7954756330000001 podStartE2EDuration="4.792723684s" totalImagesPullingTime="2.997248051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:03.790131423 +0000 UTC m=+597.283960427" watchObservedRunningTime="2026-07-21 12:21:03.792723684 +0000 UTC m=+597.286552676" Jul 21 12:21:06.949756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:06.949728 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:21:06.950917 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:06.950895 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:21:06.957730 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:06.957709 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:21:06.958986 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:06.958969 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:21:07.083739 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.083701 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd"] Jul 21 12:21:07.087106 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.087091 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.090262 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.090245 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hnnxk\"" Jul 21 12:21:07.096377 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.096357 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd"] Jul 21 12:21:07.221741 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.221645 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9kvbd\" (UID: \"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.221741 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.221733 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vjqp\" (UniqueName: \"kubernetes.io/projected/ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6-kube-api-access-9vjqp\") pod \"cert-manager-cainjector-559cdd8c87-9kvbd\" (UID: \"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.323131 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.323093 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9kvbd\" (UID: \"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.323308 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.323170 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vjqp\" (UniqueName: \"kubernetes.io/projected/ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6-kube-api-access-9vjqp\") pod \"cert-manager-cainjector-559cdd8c87-9kvbd\" (UID: \"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.333086 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.333056 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9kvbd\" (UID: \"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.333337 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.333316 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vjqp\" (UniqueName: \"kubernetes.io/projected/ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6-kube-api-access-9vjqp\") pod \"cert-manager-cainjector-559cdd8c87-9kvbd\" (UID: \"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.396542 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.396501 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" Jul 21 12:21:07.529088 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.529058 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd"] Jul 21 12:21:07.531447 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:07.531403 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae1e0dd8_73d7_43c2_a5ef_23d4e7e8f2c6.slice/crio-28ecb26849bd51d1b684e9b466f34d1e13fd1b71581ebf0963678e0f48f1305e WatchSource:0}: Error finding container 28ecb26849bd51d1b684e9b466f34d1e13fd1b71581ebf0963678e0f48f1305e: Status 404 returned error can't find the container with id 28ecb26849bd51d1b684e9b466f34d1e13fd1b71581ebf0963678e0f48f1305e Jul 21 12:21:07.790692 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.790589 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" event={"ID":"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6","Type":"ContainerStarted","Data":"2186f79485398ba22bc2c1975a65008def361117ac17c6a16584c5a0369f6c5a"} Jul 21 12:21:07.790692 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.790639 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" event={"ID":"ae1e0dd8-73d7-43c2-a5ef-23d4e7e8f2c6","Type":"ContainerStarted","Data":"28ecb26849bd51d1b684e9b466f34d1e13fd1b71581ebf0963678e0f48f1305e"} Jul 21 12:21:07.808687 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:07.808631 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9kvbd" podStartSLOduration=0.808609588 podStartE2EDuration="808.609588ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:07.807848357 +0000 UTC m=+601.301677373" watchObservedRunningTime="2026-07-21 12:21:07.808609588 +0000 UTC m=+601.302438584" Jul 21 12:21:09.777755 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:09.777725 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tgpqz" Jul 21 12:21:14.139326 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.139296 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv"] Jul 21 12:21:14.142857 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.142837 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.145746 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.145724 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 12:21:14.145852 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.145723 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:21:14.145852 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.145840 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4wt6f\"" Jul 21 12:21:14.152322 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.152293 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv"] Jul 21 12:21:14.279456 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.279375 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r28jw\" (UniqueName: \"kubernetes.io/projected/611e5e95-6cdf-4ecd-a483-dba117a82847-kube-api-access-r28jw\") pod \"openshift-lws-operator-fd8ccff4c-jg9xv\" (UID: \"611e5e95-6cdf-4ecd-a483-dba117a82847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.279617 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.279531 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/611e5e95-6cdf-4ecd-a483-dba117a82847-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jg9xv\" (UID: \"611e5e95-6cdf-4ecd-a483-dba117a82847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.380310 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.380256 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/611e5e95-6cdf-4ecd-a483-dba117a82847-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jg9xv\" (UID: \"611e5e95-6cdf-4ecd-a483-dba117a82847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.380522 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.380358 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r28jw\" (UniqueName: \"kubernetes.io/projected/611e5e95-6cdf-4ecd-a483-dba117a82847-kube-api-access-r28jw\") pod \"openshift-lws-operator-fd8ccff4c-jg9xv\" (UID: \"611e5e95-6cdf-4ecd-a483-dba117a82847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.380690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.380666 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/611e5e95-6cdf-4ecd-a483-dba117a82847-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jg9xv\" (UID: \"611e5e95-6cdf-4ecd-a483-dba117a82847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.389397 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.389342 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r28jw\" (UniqueName: \"kubernetes.io/projected/611e5e95-6cdf-4ecd-a483-dba117a82847-kube-api-access-r28jw\") pod \"openshift-lws-operator-fd8ccff4c-jg9xv\" (UID: \"611e5e95-6cdf-4ecd-a483-dba117a82847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.453201 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.453161 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" Jul 21 12:21:14.587651 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.587628 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv"] Jul 21 12:21:14.590125 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:14.590103 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod611e5e95_6cdf_4ecd_a483_dba117a82847.slice/crio-648080a69f657b43ac7eecc06662a55e780a9d6c431df01686ac59723de1cfc1 WatchSource:0}: Error finding container 648080a69f657b43ac7eecc06662a55e780a9d6c431df01686ac59723de1cfc1: Status 404 returned error can't find the container with id 648080a69f657b43ac7eecc06662a55e780a9d6c431df01686ac59723de1cfc1 Jul 21 12:21:14.828340 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:14.828310 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" event={"ID":"611e5e95-6cdf-4ecd-a483-dba117a82847","Type":"ContainerStarted","Data":"648080a69f657b43ac7eecc06662a55e780a9d6c431df01686ac59723de1cfc1"} Jul 21 12:21:17.845260 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:17.845222 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" event={"ID":"611e5e95-6cdf-4ecd-a483-dba117a82847","Type":"ContainerStarted","Data":"0057353870ab4f77c02ae24f688f88bddbff49a47712e6273d8babde0a82a5d9"} Jul 21 12:21:33.521146 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.521047 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jg9xv" podStartSLOduration=16.929121611 podStartE2EDuration="19.521012499s" totalImagesPullingTime="2.591890888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:17.868962945 +0000 UTC m=+611.362791937" watchObservedRunningTime="2026-07-21 12:21:33.521012499 +0000 UTC m=+627.014841474" Jul 21 12:21:33.522775 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.522756 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv"] Jul 21 12:21:33.526199 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.526182 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.528774 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.528753 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 12:21:33.528774 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.528756 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 12:21:33.528932 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.528876 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 12:21:33.528997 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.528977 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 12:21:33.529229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.529214 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b5mqv\"" Jul 21 12:21:33.541556 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.541533 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv"] Jul 21 12:21:33.650117 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.650077 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95006fc7-bc4a-499a-979e-ec8d0748ec43-tmp\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.650117 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.650118 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/95006fc7-bc4a-499a-979e-ec8d0748ec43-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.650340 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.650211 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95006fc7-bc4a-499a-979e-ec8d0748ec43-webhook-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.650340 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.650286 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/95006fc7-bc4a-499a-979e-ec8d0748ec43-manifests\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.650416 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.650354 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhwlw\" (UniqueName: \"kubernetes.io/projected/95006fc7-bc4a-499a-979e-ec8d0748ec43-kube-api-access-hhwlw\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.751741 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.751708 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhwlw\" (UniqueName: \"kubernetes.io/projected/95006fc7-bc4a-499a-979e-ec8d0748ec43-kube-api-access-hhwlw\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.751931 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.751756 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95006fc7-bc4a-499a-979e-ec8d0748ec43-tmp\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.751931 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.751772 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/95006fc7-bc4a-499a-979e-ec8d0748ec43-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.751931 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.751891 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95006fc7-bc4a-499a-979e-ec8d0748ec43-webhook-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.752093 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.751969 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/95006fc7-bc4a-499a-979e-ec8d0748ec43-manifests\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.752306 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.752274 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/95006fc7-bc4a-499a-979e-ec8d0748ec43-manifests\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.754004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.753978 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95006fc7-bc4a-499a-979e-ec8d0748ec43-tmp\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.754097 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.754051 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/95006fc7-bc4a-499a-979e-ec8d0748ec43-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.754204 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.754185 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95006fc7-bc4a-499a-979e-ec8d0748ec43-webhook-cert\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.762604 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.762580 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhwlw\" (UniqueName: \"kubernetes.io/projected/95006fc7-bc4a-499a-979e-ec8d0748ec43-kube-api-access-hhwlw\") pod \"opendatahub-operator-controller-manager-58f85c6b8d-rt6rv\" (UID: \"95006fc7-bc4a-499a-979e-ec8d0748ec43\") " pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.836311 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.836201 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:33.983407 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:33.983375 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv"] Jul 21 12:21:33.987112 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:33.987086 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95006fc7_bc4a_499a_979e_ec8d0748ec43.slice/crio-809da25233f0c17f202b8973b974bbad60a712812a798f23e76c0964295f6cb5 WatchSource:0}: Error finding container 809da25233f0c17f202b8973b974bbad60a712812a798f23e76c0964295f6cb5: Status 404 returned error can't find the container with id 809da25233f0c17f202b8973b974bbad60a712812a798f23e76c0964295f6cb5 Jul 21 12:21:34.924436 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:34.924397 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" event={"ID":"95006fc7-bc4a-499a-979e-ec8d0748ec43","Type":"ContainerStarted","Data":"809da25233f0c17f202b8973b974bbad60a712812a798f23e76c0964295f6cb5"} Jul 21 12:21:37.942444 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:37.942388 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" event={"ID":"95006fc7-bc4a-499a-979e-ec8d0748ec43","Type":"ContainerStarted","Data":"a0fc4a333fec014662014871ef37dc1686ffa4855e35f624870a5ffb4abcd68b"} Jul 21 12:21:38.948864 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:38.948834 2550 generic.go:356] "Generic (PLEG): container finished" podID="95006fc7-bc4a-499a-979e-ec8d0748ec43" containerID="a0fc4a333fec014662014871ef37dc1686ffa4855e35f624870a5ffb4abcd68b" exitCode=0 Jul 21 12:21:38.949252 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:38.948913 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" event={"ID":"95006fc7-bc4a-499a-979e-ec8d0748ec43","Type":"ContainerDied","Data":"a0fc4a333fec014662014871ef37dc1686ffa4855e35f624870a5ffb4abcd68b"} Jul 21 12:21:39.957467 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:39.957420 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" event={"ID":"95006fc7-bc4a-499a-979e-ec8d0748ec43","Type":"ContainerStarted","Data":"b6eb9a91fe8b28c9d312d596140691d48e8932fb48f2f96fc4792001260ba853"} Jul 21 12:21:39.957866 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:39.957586 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:40.011013 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:40.010961 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" podStartSLOduration=2.286204598 podStartE2EDuration="7.010945308s" totalImagesPullingTime="3.715786412s" totalInitContainerRuntime="1.008954298s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:40.009675663 +0000 UTC m=+633.503504656" watchObservedRunningTime="2026-07-21 12:21:40.010945308 +0000 UTC m=+633.504774300" Jul 21 12:21:50.964951 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:50.964921 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58f85c6b8d-rt6rv" Jul 21 12:21:53.896511 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:53.896480 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx"] Jul 21 12:21:53.899895 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:53.899879 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:53.902838 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:53.902820 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 12:21:53.903013 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:53.902996 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 12:21:53.929460 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:53.929418 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx"] Jul 21 12:21:54.038303 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.038270 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4531b066-13dd-4ad0-9663-edc34afb7245-tls-certs\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.038303 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.038306 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg6fq\" (UniqueName: \"kubernetes.io/projected/4531b066-13dd-4ad0-9663-edc34afb7245-kube-api-access-bg6fq\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.038577 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.038351 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4531b066-13dd-4ad0-9663-edc34afb7245-tmp\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.139800 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.139759 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg6fq\" (UniqueName: \"kubernetes.io/projected/4531b066-13dd-4ad0-9663-edc34afb7245-kube-api-access-bg6fq\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.140002 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.139842 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4531b066-13dd-4ad0-9663-edc34afb7245-tmp\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.140002 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.139920 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4531b066-13dd-4ad0-9663-edc34afb7245-tls-certs\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.142131 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.142104 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4531b066-13dd-4ad0-9663-edc34afb7245-tmp\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.142371 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.142352 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4531b066-13dd-4ad0-9663-edc34afb7245-tls-certs\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.166667 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.166594 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c5l8l"] Jul 21 12:21:54.170131 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.170111 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.173759 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.173739 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-97clt\"" Jul 21 12:21:54.173880 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.173739 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 21 12:21:54.174993 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.174970 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg6fq\" (UniqueName: \"kubernetes.io/projected/4531b066-13dd-4ad0-9663-edc34afb7245-kube-api-access-bg6fq\") pod \"kube-auth-proxy-6fc49b7cdf-vtfmx\" (UID: \"4531b066-13dd-4ad0-9663-edc34afb7245\") " pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.194192 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.194168 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c5l8l"] Jul 21 12:21:54.209410 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.209382 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" Jul 21 12:21:54.240500 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.240437 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f37db97-d0e9-434f-964f-26b04ab3b139-cert\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.240641 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.240523 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2pm2\" (UniqueName: \"kubernetes.io/projected/2f37db97-d0e9-434f-964f-26b04ab3b139-kube-api-access-k2pm2\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.341738 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.341704 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2pm2\" (UniqueName: \"kubernetes.io/projected/2f37db97-d0e9-434f-964f-26b04ab3b139-kube-api-access-k2pm2\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.341891 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.341816 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f37db97-d0e9-434f-964f-26b04ab3b139-cert\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.341944 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:21:54.341929 2550 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 21 12:21:54.342003 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:21:54.341992 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f37db97-d0e9-434f-964f-26b04ab3b139-cert podName:2f37db97-d0e9-434f-964f-26b04ab3b139 nodeName:}" failed. No retries permitted until 2026-07-21 12:21:54.84197238 +0000 UTC m=+648.335801360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f37db97-d0e9-434f-964f-26b04ab3b139-cert") pod "odh-model-controller-7998c67dc4-c5l8l" (UID: "2f37db97-d0e9-434f-964f-26b04ab3b139") : secret "odh-model-controller-webhook-cert" not found Jul 21 12:21:54.367605 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.367575 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2pm2\" (UniqueName: \"kubernetes.io/projected/2f37db97-d0e9-434f-964f-26b04ab3b139-kube-api-access-k2pm2\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.378502 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.378482 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx"] Jul 21 12:21:54.380233 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:54.380213 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4531b066_13dd_4ad0_9663_edc34afb7245.slice/crio-9aca95101cd2c94b229dbac3b692cbdf358d5253d9f86314b6095a9bb68b11b4 WatchSource:0}: Error finding container 9aca95101cd2c94b229dbac3b692cbdf358d5253d9f86314b6095a9bb68b11b4: Status 404 returned error can't find the container with id 9aca95101cd2c94b229dbac3b692cbdf358d5253d9f86314b6095a9bb68b11b4 Jul 21 12:21:54.847089 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.847056 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f37db97-d0e9-434f-964f-26b04ab3b139-cert\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:54.849468 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:54.849444 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f37db97-d0e9-434f-964f-26b04ab3b139-cert\") pod \"odh-model-controller-7998c67dc4-c5l8l\" (UID: \"2f37db97-d0e9-434f-964f-26b04ab3b139\") " pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:55.042817 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.042780 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" event={"ID":"4531b066-13dd-4ad0-9663-edc34afb7245","Type":"ContainerStarted","Data":"9aca95101cd2c94b229dbac3b692cbdf358d5253d9f86314b6095a9bb68b11b4"} Jul 21 12:21:55.090942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.090908 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:21:55.275000 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.274970 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c5l8l"] Jul 21 12:21:55.276796 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:55.276767 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f37db97_d0e9_434f_964f_26b04ab3b139.slice/crio-c3fe77b434e67b910a2184244b2d131d49033e68478a5db5eb3bc0c3b50cb75d WatchSource:0}: Error finding container c3fe77b434e67b910a2184244b2d131d49033e68478a5db5eb3bc0c3b50cb75d: Status 404 returned error can't find the container with id c3fe77b434e67b910a2184244b2d131d49033e68478a5db5eb3bc0c3b50cb75d Jul 21 12:21:55.847969 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.847931 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bstq7"] Jul 21 12:21:55.852846 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.852816 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:55.857509 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.857488 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 21 12:21:55.870130 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.870102 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bstq7"] Jul 21 12:21:55.956317 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.956280 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59vkr\" (UniqueName: \"kubernetes.io/projected/02182868-a2d3-489f-a93f-9d33d730fe57-kube-api-access-59vkr\") pod \"model-serving-api-54d89bd5fc-bstq7\" (UID: \"02182868-a2d3-489f-a93f-9d33d730fe57\") " pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:55.956536 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:55.956343 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02182868-a2d3-489f-a93f-9d33d730fe57-tls-certs\") pod \"model-serving-api-54d89bd5fc-bstq7\" (UID: \"02182868-a2d3-489f-a93f-9d33d730fe57\") " pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:56.050290 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.050210 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" event={"ID":"2f37db97-d0e9-434f-964f-26b04ab3b139","Type":"ContainerStarted","Data":"c3fe77b434e67b910a2184244b2d131d49033e68478a5db5eb3bc0c3b50cb75d"} Jul 21 12:21:56.057815 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.057731 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59vkr\" (UniqueName: \"kubernetes.io/projected/02182868-a2d3-489f-a93f-9d33d730fe57-kube-api-access-59vkr\") pod \"model-serving-api-54d89bd5fc-bstq7\" (UID: \"02182868-a2d3-489f-a93f-9d33d730fe57\") " pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:56.057962 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.057821 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02182868-a2d3-489f-a93f-9d33d730fe57-tls-certs\") pod \"model-serving-api-54d89bd5fc-bstq7\" (UID: \"02182868-a2d3-489f-a93f-9d33d730fe57\") " pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:56.061041 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.061015 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02182868-a2d3-489f-a93f-9d33d730fe57-tls-certs\") pod \"model-serving-api-54d89bd5fc-bstq7\" (UID: \"02182868-a2d3-489f-a93f-9d33d730fe57\") " pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:56.073155 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.073126 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59vkr\" (UniqueName: \"kubernetes.io/projected/02182868-a2d3-489f-a93f-9d33d730fe57-kube-api-access-59vkr\") pod \"model-serving-api-54d89bd5fc-bstq7\" (UID: \"02182868-a2d3-489f-a93f-9d33d730fe57\") " pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:56.164247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.164012 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:21:56.364207 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.363982 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5hphq"] Jul 21 12:21:56.369255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.369231 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.376061 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.375700 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bstq7"] Jul 21 12:21:56.376061 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:56.375864 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02182868_a2d3_489f_a93f_9d33d730fe57.slice/crio-0b6a4a8924ae363d7ff4e083874d788eb1648cd6600c86da639268d66c73e539 WatchSource:0}: Error finding container 0b6a4a8924ae363d7ff4e083874d788eb1648cd6600c86da639268d66c73e539: Status 404 returned error can't find the container with id 0b6a4a8924ae363d7ff4e083874d788eb1648cd6600c86da639268d66c73e539 Jul 21 12:21:56.391112 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.391087 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 12:21:56.392571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.392548 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 12:21:56.392690 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.392598 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-k2sph\"" Jul 21 12:21:56.392826 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.392809 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 12:21:56.392996 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.392981 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 12:21:56.451143 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.451110 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5hphq"] Jul 21 12:21:56.462692 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462652 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.462858 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462721 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.462858 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462759 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-cacerts\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.462858 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462800 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7hhq\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-kube-api-access-b7hhq\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.462858 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462832 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f240cb-251b-4062-9071-c6637fd0063c-local-certs\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.463070 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462881 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-istio-token\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.463070 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.462945 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564525 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564492 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564538 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-cacerts\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564570 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7hhq\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-kube-api-access-b7hhq\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564592 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f240cb-251b-4062-9071-c6637fd0063c-local-certs\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564641 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-istio-token\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564703 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.564972 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.564748 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.565359 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.565332 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.567763 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.567717 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.568255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.568214 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-cacerts\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.568523 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.568496 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.569478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.569402 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f240cb-251b-4062-9071-c6637fd0063c-local-certs\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.586165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.586104 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-istio-token\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.588634 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.588587 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7hhq\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-kube-api-access-b7hhq\") pod \"istiod-openshift-gateway-7574459656-5hphq\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:56.695727 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:56.695694 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:21:57.058226 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:57.058150 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" event={"ID":"02182868-a2d3-489f-a93f-9d33d730fe57","Type":"ContainerStarted","Data":"0b6a4a8924ae363d7ff4e083874d788eb1648cd6600c86da639268d66c73e539"} Jul 21 12:21:58.744542 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:58.744521 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5hphq"] Jul 21 12:21:58.984449 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:21:58.984391 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f240cb_251b_4062_9071_c6637fd0063c.slice/crio-c1ff87d821f245f306faa0856e72a449eb58adb821257476b971b3187218c232 WatchSource:0}: Error finding container c1ff87d821f245f306faa0856e72a449eb58adb821257476b971b3187218c232: Status 404 returned error can't find the container with id c1ff87d821f245f306faa0856e72a449eb58adb821257476b971b3187218c232 Jul 21 12:21:59.077145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:59.077007 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" event={"ID":"4531b066-13dd-4ad0-9663-edc34afb7245","Type":"ContainerStarted","Data":"2bc7f929752ef3e65ade341b0fab8e8c44fab07d96e5b7be9fa249e361a8295a"} Jul 21 12:21:59.078768 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:59.078745 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" event={"ID":"57f240cb-251b-4062-9071-c6637fd0063c","Type":"ContainerStarted","Data":"c1ff87d821f245f306faa0856e72a449eb58adb821257476b971b3187218c232"} Jul 21 12:21:59.098685 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:21:59.098632 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fc49b7cdf-vtfmx" podStartSLOduration=1.8773078170000002 podStartE2EDuration="6.098611851s" totalImagesPullingTime="4.221304034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:21:59.097595757 +0000 UTC m=+652.591424751" watchObservedRunningTime="2026-07-21 12:21:59.098611851 +0000 UTC m=+652.592440845" Jul 21 12:22:00.084740 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:00.084704 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" event={"ID":"2f37db97-d0e9-434f-964f-26b04ab3b139","Type":"ContainerStarted","Data":"113fe059bdca18c0616ea90bbf6fbc3d6827694acab73e791e8d299e1e9145b7"} Jul 21 12:22:00.085216 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:00.084814 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:22:00.086378 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:00.086352 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" event={"ID":"02182868-a2d3-489f-a93f-9d33d730fe57","Type":"ContainerStarted","Data":"3b9f4ea4b857ad30d51e49e87e92d46eaf05ad941a4c3d6843de804c005caac2"} Jul 21 12:22:00.086554 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:00.086419 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:22:00.132746 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:00.132697 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" podStartSLOduration=2.413289023 podStartE2EDuration="6.132680607s" totalImagesPullingTime="3.719391584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:00.131632594 +0000 UTC m=+653.625461598" watchObservedRunningTime="2026-07-21 12:22:00.132680607 +0000 UTC m=+653.626509648" Jul 21 12:22:00.167756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:00.167707 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" podStartSLOduration=2.509240451 podStartE2EDuration="5.16768889s" totalImagesPullingTime="2.658448439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:00.167652623 +0000 UTC m=+653.661481615" watchObservedRunningTime="2026-07-21 12:22:00.16768889 +0000 UTC m=+653.661517879" Jul 21 12:22:06.900041 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.900003 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l"] Jul 21 12:22:06.903472 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.903454 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:06.908865 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.908845 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 21 12:22:06.909000 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.908890 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ghkb4\"" Jul 21 12:22:06.920100 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.920079 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l"] Jul 21 12:22:06.947534 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.947505 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl"] Jul 21 12:22:06.952212 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.952195 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:06.956466 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.956421 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dcmt8\"" Jul 21 12:22:06.956568 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.956490 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 21 12:22:06.973126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:06.973100 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl"] Jul 21 12:22:07.068836 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.068803 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2af2e541-15f0-4b30-b60a-2891733736b4-cert\") pod \"kserve-controller-manager-d6f4b65b8-xcq5l\" (UID: \"2af2e541-15f0-4b30-b60a-2891733736b4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.069004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.068884 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6nsj\" (UniqueName: \"kubernetes.io/projected/2af2e541-15f0-4b30-b60a-2891733736b4-kube-api-access-g6nsj\") pod \"kserve-controller-manager-d6f4b65b8-xcq5l\" (UID: \"2af2e541-15f0-4b30-b60a-2891733736b4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.069004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.068901 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c55814-d746-4eeb-bd8b-815d7aff4ff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.069004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.068920 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z58vf\" (UniqueName: \"kubernetes.io/projected/19c55814-d746-4eeb-bd8b-815d7aff4ff9-kube-api-access-z58vf\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.169974 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.169888 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2af2e541-15f0-4b30-b60a-2891733736b4-cert\") pod \"kserve-controller-manager-d6f4b65b8-xcq5l\" (UID: \"2af2e541-15f0-4b30-b60a-2891733736b4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.169974 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.169962 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6nsj\" (UniqueName: \"kubernetes.io/projected/2af2e541-15f0-4b30-b60a-2891733736b4-kube-api-access-g6nsj\") pod \"kserve-controller-manager-d6f4b65b8-xcq5l\" (UID: \"2af2e541-15f0-4b30-b60a-2891733736b4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.170165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.169980 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c55814-d746-4eeb-bd8b-815d7aff4ff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.170165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.169998 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z58vf\" (UniqueName: \"kubernetes.io/projected/19c55814-d746-4eeb-bd8b-815d7aff4ff9-kube-api-access-z58vf\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.170165 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:22:07.170119 2550 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 21 12:22:07.170257 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:22:07.170181 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19c55814-d746-4eeb-bd8b-815d7aff4ff9-cert podName:19c55814-d746-4eeb-bd8b-815d7aff4ff9 nodeName:}" failed. No retries permitted until 2026-07-21 12:22:07.670165037 +0000 UTC m=+661.163994016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/19c55814-d746-4eeb-bd8b-815d7aff4ff9-cert") pod "llmisvc-controller-manager-777c6bd7d5-7lskl" (UID: "19c55814-d746-4eeb-bd8b-815d7aff4ff9") : secret "llmisvc-webhook-server-cert" not found Jul 21 12:22:07.172366 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.172338 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2af2e541-15f0-4b30-b60a-2891733736b4-cert\") pod \"kserve-controller-manager-d6f4b65b8-xcq5l\" (UID: \"2af2e541-15f0-4b30-b60a-2891733736b4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.191359 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.191323 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6nsj\" (UniqueName: \"kubernetes.io/projected/2af2e541-15f0-4b30-b60a-2891733736b4-kube-api-access-g6nsj\") pod \"kserve-controller-manager-d6f4b65b8-xcq5l\" (UID: \"2af2e541-15f0-4b30-b60a-2891733736b4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.191359 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.191348 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z58vf\" (UniqueName: \"kubernetes.io/projected/19c55814-d746-4eeb-bd8b-815d7aff4ff9-kube-api-access-z58vf\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.214061 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.214026 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:07.436463 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.436442 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l"] Jul 21 12:22:07.438387 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:22:07.438361 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af2e541_15f0_4b30_b60a_2891733736b4.slice/crio-c7abaeb742973ddd1eb903b7dd04d1aa4ff506bbaa36274526109bb9a927ecc1 WatchSource:0}: Error finding container c7abaeb742973ddd1eb903b7dd04d1aa4ff506bbaa36274526109bb9a927ecc1: Status 404 returned error can't find the container with id c7abaeb742973ddd1eb903b7dd04d1aa4ff506bbaa36274526109bb9a927ecc1 Jul 21 12:22:07.674820 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.674778 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c55814-d746-4eeb-bd8b-815d7aff4ff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.677130 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.677107 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c55814-d746-4eeb-bd8b-815d7aff4ff9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7lskl\" (UID: \"19c55814-d746-4eeb-bd8b-815d7aff4ff9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:07.861256 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:07.861173 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:08.017984 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.017924 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl"] Jul 21 12:22:08.020220 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:22:08.020191 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod19c55814_d746_4eeb_bd8b_815d7aff4ff9.slice/crio-d5cd1f0d274375d6df7c346f5ba1f80b8180456600b6b4a2e3f5b11559cd60cb WatchSource:0}: Error finding container d5cd1f0d274375d6df7c346f5ba1f80b8180456600b6b4a2e3f5b11559cd60cb: Status 404 returned error can't find the container with id d5cd1f0d274375d6df7c346f5ba1f80b8180456600b6b4a2e3f5b11559cd60cb Jul 21 12:22:08.129346 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.129254 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" event={"ID":"2af2e541-15f0-4b30-b60a-2891733736b4","Type":"ContainerStarted","Data":"c7abaeb742973ddd1eb903b7dd04d1aa4ff506bbaa36274526109bb9a927ecc1"} Jul 21 12:22:08.130889 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.130848 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" event={"ID":"19c55814-d746-4eeb-bd8b-815d7aff4ff9","Type":"ContainerStarted","Data":"d5cd1f0d274375d6df7c346f5ba1f80b8180456600b6b4a2e3f5b11559cd60cb"} Jul 21 12:22:08.863420 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.863387 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799"] Jul 21 12:22:08.866701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.866683 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.872974 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.872950 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zmkvt\"" Jul 21 12:22:08.896530 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.896498 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799"] Jul 21 12:22:08.987380 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987329 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qrg\" (UniqueName: \"kubernetes.io/projected/e61b7390-f5e5-4c3c-95cb-2362d817361e-kube-api-access-q5qrg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987389 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987416 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987466 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987553 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987581 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987831 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987617 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987831 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987664 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:08.987831 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:08.987688 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b7390-f5e5-4c3c-95cb-2362d817361e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.088859 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.088820 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.088872 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089017 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089095 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089132 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b7390-f5e5-4c3c-95cb-2362d817361e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089212 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qrg\" (UniqueName: \"kubernetes.io/projected/e61b7390-f5e5-4c3c-95cb-2362d817361e-kube-api-access-q5qrg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089246 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089269 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089274 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089301 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089825 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089498 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089825 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089512 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.089825 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.089766 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.090195 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.090173 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e61b7390-f5e5-4c3c-95cb-2362d817361e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.092000 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.091973 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.092352 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.092333 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.147504 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.147409 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qrg\" (UniqueName: \"kubernetes.io/projected/e61b7390-f5e5-4c3c-95cb-2362d817361e-kube-api-access-q5qrg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.152302 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.152277 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e61b7390-f5e5-4c3c-95cb-2362d817361e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wq799\" (UID: \"e61b7390-f5e5-4c3c-95cb-2362d817361e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:09.178644 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:09.178599 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:10.600207 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:10.600161 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799"] Jul 21 12:22:10.628005 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:22:10.627971 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61b7390_f5e5_4c3c_95cb_2362d817361e.slice/crio-32f3335056f665a8fb5c8a9a565fdf56b95a8bd6dede03c1d366ad2a1e493b95 WatchSource:0}: Error finding container 32f3335056f665a8fb5c8a9a565fdf56b95a8bd6dede03c1d366ad2a1e493b95: Status 404 returned error can't find the container with id 32f3335056f665a8fb5c8a9a565fdf56b95a8bd6dede03c1d366ad2a1e493b95 Jul 21 12:22:10.987817 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:10.987781 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5hphq"] Jul 21 12:22:11.094853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:11.094715 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-c5l8l" Jul 21 12:22:11.097504 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:11.097477 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bstq7" Jul 21 12:22:11.149488 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:11.149416 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" event={"ID":"e61b7390-f5e5-4c3c-95cb-2362d817361e","Type":"ContainerStarted","Data":"32f3335056f665a8fb5c8a9a565fdf56b95a8bd6dede03c1d366ad2a1e493b95"} Jul 21 12:22:12.511837 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:12.511794 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:22:12.512203 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:12.511872 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:22:13.163765 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.163473 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" event={"ID":"57f240cb-251b-4062-9071-c6637fd0063c","Type":"ContainerStarted","Data":"2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430"} Jul 21 12:22:13.163765 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.163539 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:22:13.163765 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.163557 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" podUID="57f240cb-251b-4062-9071-c6637fd0063c" containerName="discovery" containerID="cri-o://2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430" gracePeriod=30 Jul 21 12:22:13.165544 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.165515 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" event={"ID":"19c55814-d746-4eeb-bd8b-815d7aff4ff9","Type":"ContainerStarted","Data":"aa8d7419740ddb65b15ae8706402597cc8142e78fc490bf60baa97698ff1a18f"} Jul 21 12:22:13.165684 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.165589 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:13.165867 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.165839 2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5hphq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 12:22:13.165984 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.165888 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" podUID="57f240cb-251b-4062-9071-c6637fd0063c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 12:22:13.167876 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.167830 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" event={"ID":"2af2e541-15f0-4b30-b60a-2891733736b4","Type":"ContainerStarted","Data":"7443465d37b7fab9c94f365ccd8c832b5415d6b735172a060ddb46dbaa66768c"} Jul 21 12:22:13.168013 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.167995 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:13.188955 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.188905 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" podStartSLOduration=3.665327354 podStartE2EDuration="17.18888352s" totalImagesPullingTime="13.523556166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:13.187460813 +0000 UTC m=+666.681289808" watchObservedRunningTime="2026-07-21 12:22:13.18888352 +0000 UTC m=+666.682712514" Jul 21 12:22:13.207004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.206954 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" podStartSLOduration=2.717113707 podStartE2EDuration="7.206937538s" totalImagesPullingTime="4.489823831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:13.204924691 +0000 UTC m=+666.698753684" watchObservedRunningTime="2026-07-21 12:22:13.206937538 +0000 UTC m=+666.700766532" Jul 21 12:22:13.228572 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.228513 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" podStartSLOduration=4.042727963 podStartE2EDuration="7.228489272s" totalImagesPullingTime="3.185761309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:13.225839489 +0000 UTC m=+666.719668489" watchObservedRunningTime="2026-07-21 12:22:13.228489272 +0000 UTC m=+666.722318259" Jul 21 12:22:13.444664 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.444619 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:22:13.534914 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.534883 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f240cb-251b-4062-9071-c6637fd0063c-local-certs\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.534966 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-dns-cert\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535001 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-istio-token\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535029 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-cacerts\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535063 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-ca-configmap\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535237 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-kubeconfig\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535333 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535280 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7hhq\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-kube-api-access-b7hhq\") pod \"57f240cb-251b-4062-9071-c6637fd0063c\" (UID: \"57f240cb-251b-4062-9071-c6637fd0063c\") " Jul 21 12:22:13.535727 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535414 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:22:13.535727 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.535646 2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-ca-configmap\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:13.537734 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.537708 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-cacerts" (OuterVolumeSpecName: "cacerts") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:22:13.537734 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.537730 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-istio-token" (OuterVolumeSpecName: "istio-token") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:22:13.538375 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.538341 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:22:13.538375 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.538360 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f240cb-251b-4062-9071-c6637fd0063c-local-certs" (OuterVolumeSpecName: "local-certs") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:22:13.538535 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.538406 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-kube-api-access-b7hhq" (OuterVolumeSpecName: "kube-api-access-b7hhq") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "kube-api-access-b7hhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:22:13.538535 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.538487 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "57f240cb-251b-4062-9071-c6637fd0063c" (UID: "57f240cb-251b-4062-9071-c6637fd0063c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:22:13.636834 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.636798 2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-csr-dns-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:13.636834 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.636830 2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-istio-token\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:13.636834 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.636842 2550 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-cacerts\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:13.637055 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.636854 2550 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57f240cb-251b-4062-9071-c6637fd0063c-istio-kubeconfig\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:13.637055 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.636866 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7hhq\" (UniqueName: \"kubernetes.io/projected/57f240cb-251b-4062-9071-c6637fd0063c-kube-api-access-b7hhq\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:13.637055 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:13.636878 2550 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57f240cb-251b-4062-9071-c6637fd0063c-local-certs\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:14.175955 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.175921 2550 generic.go:356] "Generic (PLEG): container finished" podID="57f240cb-251b-4062-9071-c6637fd0063c" containerID="2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430" exitCode=2 Jul 21 12:22:14.176128 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.175966 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" Jul 21 12:22:14.176128 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.176017 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" event={"ID":"57f240cb-251b-4062-9071-c6637fd0063c","Type":"ContainerDied","Data":"2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430"} Jul 21 12:22:14.176128 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.176079 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5hphq" event={"ID":"57f240cb-251b-4062-9071-c6637fd0063c","Type":"ContainerDied","Data":"c1ff87d821f245f306faa0856e72a449eb58adb821257476b971b3187218c232"} Jul 21 12:22:14.176128 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.176103 2550 scope.go:117] "RemoveContainer" containerID="2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430" Jul 21 12:22:14.205180 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.205152 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5hphq"] Jul 21 12:22:14.209802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.209768 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5hphq"] Jul 21 12:22:14.512573 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.512551 2550 scope.go:117] "RemoveContainer" containerID="2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430" Jul 21 12:22:14.512909 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:22:14.512890 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430\": container with ID starting with 2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430 not found: ID does not exist" containerID="2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430" Jul 21 12:22:14.512950 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.512919 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430"} err="failed to get container status \"2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430\": rpc error: code = NotFound desc = could not find container \"2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430\": container with ID starting with 2e2517c1a586a6b0c3ce0c87c516021ea65d32637be21f9ca037bc16c9bdf430 not found: ID does not exist" Jul 21 12:22:14.573007 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.572974 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:22:14.573313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.573053 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:22:14.573313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:14.573084 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:22:15.120493 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:15.120450 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f240cb-251b-4062-9071-c6637fd0063c" path="/var/lib/kubelet/pods/57f240cb-251b-4062-9071-c6637fd0063c/volumes" Jul 21 12:22:15.183367 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:15.183338 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" event={"ID":"e61b7390-f5e5-4c3c-95cb-2362d817361e","Type":"ContainerStarted","Data":"acb6fee36c4a1170204ccd351f584daf4ecd78cdcd5bb4cc44410bbb4c103936"} Jul 21 12:22:15.209260 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:15.209212 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" podStartSLOduration=3.26731961 podStartE2EDuration="7.20919736s" totalImagesPullingTime="3.94187775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:15.208854529 +0000 UTC m=+668.702683524" watchObservedRunningTime="2026-07-21 12:22:15.20919736 +0000 UTC m=+668.703026354" Jul 21 12:22:16.179570 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:16.179539 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:16.184043 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:16.184018 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:16.188265 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:16.188235 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:16.188976 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:16.188957 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wq799" Jul 21 12:22:44.179332 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:44.179302 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7lskl" Jul 21 12:22:44.182749 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:44.182727 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq5l" Jul 21 12:22:54.502307 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.502274 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w55vf"] Jul 21 12:22:54.502715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.502666 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f240cb-251b-4062-9071-c6637fd0063c" containerName="discovery" Jul 21 12:22:54.502715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.502677 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f240cb-251b-4062-9071-c6637fd0063c" containerName="discovery" Jul 21 12:22:54.502791 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.502754 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f240cb-251b-4062-9071-c6637fd0063c" containerName="discovery" Jul 21 12:22:54.515378 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.515345 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:54.516243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.516216 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w55vf"] Jul 21 12:22:54.518064 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.518038 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 12:22:54.519352 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.519313 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 12:22:54.519352 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.519345 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cqvxm\"" Jul 21 12:22:54.624692 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.624657 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw54g\" (UniqueName: \"kubernetes.io/projected/73fa1245-69be-462d-ab7c-6d721161b46e-kube-api-access-hw54g\") pod \"kuadrant-operator-catalog-w55vf\" (UID: \"73fa1245-69be-462d-ab7c-6d721161b46e\") " pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:54.726003 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.725965 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw54g\" (UniqueName: \"kubernetes.io/projected/73fa1245-69be-462d-ab7c-6d721161b46e-kube-api-access-hw54g\") pod \"kuadrant-operator-catalog-w55vf\" (UID: \"73fa1245-69be-462d-ab7c-6d721161b46e\") " pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:54.734835 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.734806 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw54g\" (UniqueName: \"kubernetes.io/projected/73fa1245-69be-462d-ab7c-6d721161b46e-kube-api-access-hw54g\") pod \"kuadrant-operator-catalog-w55vf\" (UID: \"73fa1245-69be-462d-ab7c-6d721161b46e\") " pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:54.825274 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.825187 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:54.963393 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:54.963365 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w55vf"] Jul 21 12:22:54.964954 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:22:54.964927 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fa1245_69be_462d_ab7c_6d721161b46e.slice/crio-02c3826ee509a5efdf7a94280e421117a6cb2f596bd570f829632b7215c7fe7a WatchSource:0}: Error finding container 02c3826ee509a5efdf7a94280e421117a6cb2f596bd570f829632b7215c7fe7a: Status 404 returned error can't find the container with id 02c3826ee509a5efdf7a94280e421117a6cb2f596bd570f829632b7215c7fe7a Jul 21 12:22:55.076245 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.076159 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w55vf"] Jul 21 12:22:55.092823 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.092796 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lxbd"] Jul 21 12:22:55.097517 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.097501 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:22:55.104655 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.104629 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lxbd"] Jul 21 12:22:55.129896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.129868 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g74zc\" (UniqueName: \"kubernetes.io/projected/841d08d0-4a98-40a4-b817-92ee60119023-kube-api-access-g74zc\") pod \"kuadrant-operator-catalog-4lxbd\" (UID: \"841d08d0-4a98-40a4-b817-92ee60119023\") " pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:22:55.230286 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.230255 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g74zc\" (UniqueName: \"kubernetes.io/projected/841d08d0-4a98-40a4-b817-92ee60119023-kube-api-access-g74zc\") pod \"kuadrant-operator-catalog-4lxbd\" (UID: \"841d08d0-4a98-40a4-b817-92ee60119023\") " pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:22:55.239735 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.239704 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g74zc\" (UniqueName: \"kubernetes.io/projected/841d08d0-4a98-40a4-b817-92ee60119023-kube-api-access-g74zc\") pod \"kuadrant-operator-catalog-4lxbd\" (UID: \"841d08d0-4a98-40a4-b817-92ee60119023\") " pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:22:55.378018 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.377933 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" event={"ID":"73fa1245-69be-462d-ab7c-6d721161b46e","Type":"ContainerStarted","Data":"02c3826ee509a5efdf7a94280e421117a6cb2f596bd570f829632b7215c7fe7a"} Jul 21 12:22:55.408346 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.408324 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:22:55.549377 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:55.549353 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lxbd"] Jul 21 12:22:55.575171 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:22:55.575140 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841d08d0_4a98_40a4_b817_92ee60119023.slice/crio-993f1cd3786d00abe11a6dd939eaf4469888b722bfe5d4e4f1d9c3772ebf825c WatchSource:0}: Error finding container 993f1cd3786d00abe11a6dd939eaf4469888b722bfe5d4e4f1d9c3772ebf825c: Status 404 returned error can't find the container with id 993f1cd3786d00abe11a6dd939eaf4469888b722bfe5d4e4f1d9c3772ebf825c Jul 21 12:22:56.385814 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:56.385777 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" event={"ID":"841d08d0-4a98-40a4-b817-92ee60119023","Type":"ContainerStarted","Data":"993f1cd3786d00abe11a6dd939eaf4469888b722bfe5d4e4f1d9c3772ebf825c"} Jul 21 12:22:57.391826 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.391735 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" event={"ID":"73fa1245-69be-462d-ab7c-6d721161b46e","Type":"ContainerStarted","Data":"66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308"} Jul 21 12:22:57.391826 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.391814 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" podUID="73fa1245-69be-462d-ab7c-6d721161b46e" containerName="registry-server" containerID="cri-o://66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308" gracePeriod=2 Jul 21 12:22:57.393233 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.393209 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" event={"ID":"841d08d0-4a98-40a4-b817-92ee60119023","Type":"ContainerStarted","Data":"5b4e4ee2417e0ee67921f2dcc4915ca045b71f176ddebc8a8a741a023f6219fe"} Jul 21 12:22:57.411675 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.411627 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" podStartSLOduration=1.421831512 podStartE2EDuration="3.411612237s" totalImagesPullingTime="1.989780725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:57.409589308 +0000 UTC m=+710.903418303" watchObservedRunningTime="2026-07-21 12:22:57.411612237 +0000 UTC m=+710.905441230" Jul 21 12:22:57.425509 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.425465 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" podStartSLOduration=1.045292897 podStartE2EDuration="2.425449549s" totalImagesPullingTime="1.380156652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:22:57.42439781 +0000 UTC m=+710.918226827" watchObservedRunningTime="2026-07-21 12:22:57.425449549 +0000 UTC m=+710.919278534" Jul 21 12:22:57.638296 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.638270 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:57.653319 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.653261 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw54g\" (UniqueName: \"kubernetes.io/projected/73fa1245-69be-462d-ab7c-6d721161b46e-kube-api-access-hw54g\") pod \"73fa1245-69be-462d-ab7c-6d721161b46e\" (UID: \"73fa1245-69be-462d-ab7c-6d721161b46e\") " Jul 21 12:22:57.655761 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.655731 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73fa1245-69be-462d-ab7c-6d721161b46e-kube-api-access-hw54g" (OuterVolumeSpecName: "kube-api-access-hw54g") pod "73fa1245-69be-462d-ab7c-6d721161b46e" (UID: "73fa1245-69be-462d-ab7c-6d721161b46e"). InnerVolumeSpecName "kube-api-access-hw54g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:22:57.754389 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:57.754333 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw54g\" (UniqueName: \"kubernetes.io/projected/73fa1245-69be-462d-ab7c-6d721161b46e-kube-api-access-hw54g\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:22:58.399243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.399131 2550 generic.go:356] "Generic (PLEG): container finished" podID="73fa1245-69be-462d-ab7c-6d721161b46e" containerID="66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308" exitCode=0 Jul 21 12:22:58.399243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.399176 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" Jul 21 12:22:58.399243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.399225 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" event={"ID":"73fa1245-69be-462d-ab7c-6d721161b46e","Type":"ContainerDied","Data":"66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308"} Jul 21 12:22:58.399815 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.399270 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w55vf" event={"ID":"73fa1245-69be-462d-ab7c-6d721161b46e","Type":"ContainerDied","Data":"02c3826ee509a5efdf7a94280e421117a6cb2f596bd570f829632b7215c7fe7a"} Jul 21 12:22:58.399815 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.399292 2550 scope.go:117] "RemoveContainer" containerID="66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308" Jul 21 12:22:58.409324 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.409308 2550 scope.go:117] "RemoveContainer" containerID="66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308" Jul 21 12:22:58.409609 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:22:58.409592 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308\": container with ID starting with 66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308 not found: ID does not exist" containerID="66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308" Jul 21 12:22:58.409686 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.409617 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308"} err="failed to get container status \"66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308\": rpc error: code = NotFound desc = could not find container \"66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308\": container with ID starting with 66673ce8335eeef0138b47ab562f1a6602b3100b4bc1171b1f318ae7e8f10308 not found: ID does not exist" Jul 21 12:22:58.420387 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.420365 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w55vf"] Jul 21 12:22:58.424336 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:58.424316 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w55vf"] Jul 21 12:22:59.120145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:22:59.120113 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73fa1245-69be-462d-ab7c-6d721161b46e" path="/var/lib/kubelet/pods/73fa1245-69be-462d-ab7c-6d721161b46e/volumes" Jul 21 12:23:05.408934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:05.408892 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:23:05.408934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:05.408947 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:23:05.432836 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:05.432803 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:23:05.457367 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:05.457341 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4lxbd" Jul 21 12:23:17.260893 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.260857 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4"] Jul 21 12:23:17.261354 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.261275 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fa1245-69be-462d-ab7c-6d721161b46e" containerName="registry-server" Jul 21 12:23:17.261354 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.261291 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fa1245-69be-462d-ab7c-6d721161b46e" containerName="registry-server" Jul 21 12:23:17.261460 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.261373 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fa1245-69be-462d-ab7c-6d721161b46e" containerName="registry-server" Jul 21 12:23:17.265742 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.265719 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:17.269965 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.269939 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-952wx\"" Jul 21 12:23:17.270478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.270458 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 21 12:23:17.278129 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.278107 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4"] Jul 21 12:23:17.325612 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.325580 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4db66\" (UniqueName: \"kubernetes.io/projected/8e6755aa-5c16-42ec-ab43-ccc580fa6bc4-kube-api-access-4db66\") pod \"dns-operator-controller-manager-864dd78c4c-gbzl4\" (UID: \"8e6755aa-5c16-42ec-ab43-ccc580fa6bc4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:17.426314 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.426278 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4db66\" (UniqueName: \"kubernetes.io/projected/8e6755aa-5c16-42ec-ab43-ccc580fa6bc4-kube-api-access-4db66\") pod \"dns-operator-controller-manager-864dd78c4c-gbzl4\" (UID: \"8e6755aa-5c16-42ec-ab43-ccc580fa6bc4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:17.439222 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.439195 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4db66\" (UniqueName: \"kubernetes.io/projected/8e6755aa-5c16-42ec-ab43-ccc580fa6bc4-kube-api-access-4db66\") pod \"dns-operator-controller-manager-864dd78c4c-gbzl4\" (UID: \"8e6755aa-5c16-42ec-ab43-ccc580fa6bc4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:17.578291 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.578205 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:17.715994 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:17.715969 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4"] Jul 21 12:23:17.716908 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:17.716881 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6755aa_5c16_42ec_ab43_ccc580fa6bc4.slice/crio-c760c1ab5e4811ca277926eeb2890cb19d61f71dee3a12142e2413df254aebe6 WatchSource:0}: Error finding container c760c1ab5e4811ca277926eeb2890cb19d61f71dee3a12142e2413df254aebe6: Status 404 returned error can't find the container with id c760c1ab5e4811ca277926eeb2890cb19d61f71dee3a12142e2413df254aebe6 Jul 21 12:23:18.498647 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:18.498613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" event={"ID":"8e6755aa-5c16-42ec-ab43-ccc580fa6bc4","Type":"ContainerStarted","Data":"c760c1ab5e4811ca277926eeb2890cb19d61f71dee3a12142e2413df254aebe6"} Jul 21 12:23:19.713356 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.713128 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797b978cdd-9wfzz"] Jul 21 12:23:19.720081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.720055 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.721200 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.721176 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797b978cdd-9wfzz"] Jul 21 12:23:19.853358 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853321 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-console-config\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.853567 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853365 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k4w9\" (UniqueName: \"kubernetes.io/projected/fb8ef86b-1dff-4409-b607-d84c695fb765-kube-api-access-6k4w9\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.853567 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853385 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-trusted-ca-bundle\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.853567 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853405 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-service-ca\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.853567 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853539 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb8ef86b-1dff-4409-b607-d84c695fb765-console-oauth-config\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.853750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853588 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-oauth-serving-cert\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.853750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.853661 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb8ef86b-1dff-4409-b607-d84c695fb765-console-serving-cert\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.954959 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.954921 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-console-config\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.954966 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k4w9\" (UniqueName: \"kubernetes.io/projected/fb8ef86b-1dff-4409-b607-d84c695fb765-kube-api-access-6k4w9\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.954984 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-trusted-ca-bundle\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955014 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-service-ca\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955320 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955231 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb8ef86b-1dff-4409-b607-d84c695fb765-console-oauth-config\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955320 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955309 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-oauth-serving-cert\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955443 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955390 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb8ef86b-1dff-4409-b607-d84c695fb765-console-serving-cert\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955870 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955841 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-console-config\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955991 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955844 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-service-ca\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.955991 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.955979 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-trusted-ca-bundle\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.956106 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.956007 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb8ef86b-1dff-4409-b607-d84c695fb765-oauth-serving-cert\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.957830 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.957807 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb8ef86b-1dff-4409-b607-d84c695fb765-console-serving-cert\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.957930 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.957913 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb8ef86b-1dff-4409-b607-d84c695fb765-console-oauth-config\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:19.965485 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:19.965443 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k4w9\" (UniqueName: \"kubernetes.io/projected/fb8ef86b-1dff-4409-b607-d84c695fb765-kube-api-access-6k4w9\") pod \"console-797b978cdd-9wfzz\" (UID: \"fb8ef86b-1dff-4409-b607-d84c695fb765\") " pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:20.034339 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.034304 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:20.181802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.181781 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797b978cdd-9wfzz"] Jul 21 12:23:20.183688 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:20.183667 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8ef86b_1dff_4409_b607_d84c695fb765.slice/crio-9dfbca060a6d4ecd919593cd08a5510f0e1296565da041132bcdb84aa7908790 WatchSource:0}: Error finding container 9dfbca060a6d4ecd919593cd08a5510f0e1296565da041132bcdb84aa7908790: Status 404 returned error can't find the container with id 9dfbca060a6d4ecd919593cd08a5510f0e1296565da041132bcdb84aa7908790 Jul 21 12:23:20.510945 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.510854 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" event={"ID":"8e6755aa-5c16-42ec-ab43-ccc580fa6bc4","Type":"ContainerStarted","Data":"17ef70bdeb774205ab2053c12691e1d9d5a9df3155147d165c96a7e446977357"} Jul 21 12:23:20.511116 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.510985 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:20.512439 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.512406 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797b978cdd-9wfzz" event={"ID":"fb8ef86b-1dff-4409-b607-d84c695fb765","Type":"ContainerStarted","Data":"8dafbbcff526b53b753866c1d290b3cff9cc35f2dfaf595addd9426af8f4e8ce"} Jul 21 12:23:20.512552 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.512451 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797b978cdd-9wfzz" event={"ID":"fb8ef86b-1dff-4409-b607-d84c695fb765","Type":"ContainerStarted","Data":"9dfbca060a6d4ecd919593cd08a5510f0e1296565da041132bcdb84aa7908790"} Jul 21 12:23:20.531275 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.531230 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" podStartSLOduration=1.591860785 podStartE2EDuration="3.531216387s" totalImagesPullingTime="1.939355602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:20.530107047 +0000 UTC m=+734.023936039" watchObservedRunningTime="2026-07-21 12:23:20.531216387 +0000 UTC m=+734.025045379" Jul 21 12:23:20.553139 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.553090 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797b978cdd-9wfzz" podStartSLOduration=1.553073879 podStartE2EDuration="1.553073879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:20.551630826 +0000 UTC m=+734.045459819" watchObservedRunningTime="2026-07-21 12:23:20.553073879 +0000 UTC m=+734.046902871" Jul 21 12:23:20.942168 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.942136 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx"] Jul 21 12:23:20.945779 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.945764 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:20.948474 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.948453 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-862zz\"" Jul 21 12:23:20.956837 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:20.956816 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx"] Jul 21 12:23:21.066518 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:21.066483 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lll9\" (UniqueName: \"kubernetes.io/projected/8668e119-47a0-4cd3-9604-54d4a490b155-kube-api-access-4lll9\") pod \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" (UID: \"8668e119-47a0-4cd3-9604-54d4a490b155\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:21.167979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:21.167946 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lll9\" (UniqueName: \"kubernetes.io/projected/8668e119-47a0-4cd3-9604-54d4a490b155-kube-api-access-4lll9\") pod \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" (UID: \"8668e119-47a0-4cd3-9604-54d4a490b155\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:21.181564 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:21.181535 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lll9\" (UniqueName: \"kubernetes.io/projected/8668e119-47a0-4cd3-9604-54d4a490b155-kube-api-access-4lll9\") pod \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" (UID: \"8668e119-47a0-4cd3-9604-54d4a490b155\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:21.257260 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:21.257168 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:21.399303 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:21.399278 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx"] Jul 21 12:23:21.401309 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:21.401285 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8668e119_47a0_4cd3_9604_54d4a490b155.slice/crio-e5b1f1862c81cd207de99b40be21a5d35a044b586aaf87ebed1751eb6a349b14 WatchSource:0}: Error finding container e5b1f1862c81cd207de99b40be21a5d35a044b586aaf87ebed1751eb6a349b14: Status 404 returned error can't find the container with id e5b1f1862c81cd207de99b40be21a5d35a044b586aaf87ebed1751eb6a349b14 Jul 21 12:23:21.518290 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:21.518208 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" event={"ID":"8668e119-47a0-4cd3-9604-54d4a490b155","Type":"ContainerStarted","Data":"e5b1f1862c81cd207de99b40be21a5d35a044b586aaf87ebed1751eb6a349b14"} Jul 21 12:23:24.536118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:24.536075 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" event={"ID":"8668e119-47a0-4cd3-9604-54d4a490b155","Type":"ContainerStarted","Data":"d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65"} Jul 21 12:23:24.536511 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:24.536138 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:24.556874 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:24.556825 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" podStartSLOduration=2.325639313 podStartE2EDuration="4.556808124s" totalImagesPullingTime="2.231168811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:24.554199527 +0000 UTC m=+738.048028520" watchObservedRunningTime="2026-07-21 12:23:24.556808124 +0000 UTC m=+738.050637117" Jul 21 12:23:25.653802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.653767 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7"] Jul 21 12:23:25.657891 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.657875 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.660320 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.660304 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ns8wt\"" Jul 21 12:23:25.671851 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.671828 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7"] Jul 21 12:23:25.813867 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.813828 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057dc58c-b142-4264-8ddb-d4cb3d310aed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.814053 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.813885 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8vp\" (UniqueName: \"kubernetes.io/projected/057dc58c-b142-4264-8ddb-d4cb3d310aed-kube-api-access-2l8vp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.915017 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.914933 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057dc58c-b142-4264-8ddb-d4cb3d310aed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.915017 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.915001 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8vp\" (UniqueName: \"kubernetes.io/projected/057dc58c-b142-4264-8ddb-d4cb3d310aed-kube-api-access-2l8vp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.915402 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.915384 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057dc58c-b142-4264-8ddb-d4cb3d310aed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.930570 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.930535 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8vp\" (UniqueName: \"kubernetes.io/projected/057dc58c-b142-4264-8ddb-d4cb3d310aed-kube-api-access-2l8vp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:25.968573 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:25.968539 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:26.119043 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:26.119019 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7"] Jul 21 12:23:26.121058 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:26.121033 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057dc58c_b142_4264_8ddb_d4cb3d310aed.slice/crio-c75bf69fc84acc1c3aa30d6b05989293fca8f38e00c1c090a46dce83ef2c7c14 WatchSource:0}: Error finding container c75bf69fc84acc1c3aa30d6b05989293fca8f38e00c1c090a46dce83ef2c7c14: Status 404 returned error can't find the container with id c75bf69fc84acc1c3aa30d6b05989293fca8f38e00c1c090a46dce83ef2c7c14 Jul 21 12:23:26.547098 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:26.547070 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" event={"ID":"057dc58c-b142-4264-8ddb-d4cb3d310aed","Type":"ContainerStarted","Data":"c75bf69fc84acc1c3aa30d6b05989293fca8f38e00c1c090a46dce83ef2c7c14"} Jul 21 12:23:29.667120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.667087 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z774d"] Jul 21 12:23:29.671003 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.670984 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:29.675216 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.675158 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bl47h\"" Jul 21 12:23:29.680752 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.680728 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z774d"] Jul 21 12:23:29.750704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.750668 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6kx\" (UniqueName: \"kubernetes.io/projected/4c6589f7-d18d-4d0e-8072-eab93d368549-kube-api-access-tt6kx\") pod \"authorino-operator-75cc8d5cb7-z774d\" (UID: \"4c6589f7-d18d-4d0e-8072-eab93d368549\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:29.852182 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.852146 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6kx\" (UniqueName: \"kubernetes.io/projected/4c6589f7-d18d-4d0e-8072-eab93d368549-kube-api-access-tt6kx\") pod \"authorino-operator-75cc8d5cb7-z774d\" (UID: \"4c6589f7-d18d-4d0e-8072-eab93d368549\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:29.866219 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.866191 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6kx\" (UniqueName: \"kubernetes.io/projected/4c6589f7-d18d-4d0e-8072-eab93d368549-kube-api-access-tt6kx\") pod \"authorino-operator-75cc8d5cb7-z774d\" (UID: \"4c6589f7-d18d-4d0e-8072-eab93d368549\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:29.984212 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:29.984170 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:30.034699 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:30.034661 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:30.034881 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:30.034733 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:30.040370 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:30.040347 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:30.574162 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:30.574116 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-797b978cdd-9wfzz" Jul 21 12:23:30.636293 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:30.636262 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf7b8cdd6-swqnk"] Jul 21 12:23:31.228638 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:31.228617 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z774d"] Jul 21 12:23:31.232640 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:31.232615 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6589f7_d18d_4d0e_8072_eab93d368549.slice/crio-a8504f983777b0ac0bef67099cc530aef5530537ee2cb6fbc2c324523226d0a4 WatchSource:0}: Error finding container a8504f983777b0ac0bef67099cc530aef5530537ee2cb6fbc2c324523226d0a4: Status 404 returned error can't find the container with id a8504f983777b0ac0bef67099cc530aef5530537ee2cb6fbc2c324523226d0a4 Jul 21 12:23:31.521243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:31.521211 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gbzl4" Jul 21 12:23:31.577195 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:31.577157 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" event={"ID":"057dc58c-b142-4264-8ddb-d4cb3d310aed","Type":"ContainerStarted","Data":"00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8"} Jul 21 12:23:31.577385 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:31.577224 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:31.578491 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:31.578463 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" event={"ID":"4c6589f7-d18d-4d0e-8072-eab93d368549","Type":"ContainerStarted","Data":"a8504f983777b0ac0bef67099cc530aef5530537ee2cb6fbc2c324523226d0a4"} Jul 21 12:23:31.600060 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:31.600015 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" podStartSLOduration=1.564257525 podStartE2EDuration="6.600000584s" totalImagesPullingTime="5.035743059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:31.597328072 +0000 UTC m=+745.091157068" watchObservedRunningTime="2026-07-21 12:23:31.600000584 +0000 UTC m=+745.093829577" Jul 21 12:23:34.598956 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:34.598926 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" event={"ID":"4c6589f7-d18d-4d0e-8072-eab93d368549","Type":"ContainerStarted","Data":"cff6c47fbd09cf0595ef0aec9d7e2497c59a5c2d73824252c9aef93d65eb1e03"} Jul 21 12:23:34.599331 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:34.598996 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:34.624034 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:34.623982 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" podStartSLOduration=3.288487539 podStartE2EDuration="5.623967104s" totalImagesPullingTime="2.335479565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:34.622613248 +0000 UTC m=+748.116442240" watchObservedRunningTime="2026-07-21 12:23:34.623967104 +0000 UTC m=+748.117796097" Jul 21 12:23:35.543955 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:35.543925 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:42.588623 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:42.588586 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.239098 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.239060 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7"] Jul 21 12:23:44.239521 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.239245 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" containerName="manager" containerID="cri-o://00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8" gracePeriod=2 Jul 21 12:23:44.247290 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.247247 2550 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"057dc58c-b142-4264-8ddb-d4cb3d310aed\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\": pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.248525 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.248500 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7"] Jul 21 12:23:44.259504 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.259480 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx"] Jul 21 12:23:44.259734 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.259688 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" containerName="manager" containerID="cri-o://d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65" gracePeriod=2 Jul 21 12:23:44.263637 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.263614 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss"] Jul 21 12:23:44.264025 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.264012 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" containerName="manager" Jul 21 12:23:44.264081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.264027 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" containerName="manager" Jul 21 12:23:44.264118 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.264093 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" containerName="manager" Jul 21 12:23:44.267284 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.267266 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.271266 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.271245 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx"] Jul 21 12:23:44.278490 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.278469 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss"] Jul 21 12:23:44.279567 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.279524 2550 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8668e119-47a0-4cd3-9604-54d4a490b155\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-bbgkx\": pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.281889 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.281862 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.284135 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.284110 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.289874 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.289850 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdwjh\" (UniqueName: \"kubernetes.io/projected/f3a5f8ef-286f-46d4-9516-77a0a04975d1-kube-api-access-gdwjh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdtss\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.290038 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.290018 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f8ef-286f-46d4-9516-77a0a04975d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdtss\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.292033 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.292001 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs"] Jul 21 12:23:44.292443 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.292415 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" containerName="manager" Jul 21 12:23:44.292443 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.292441 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" containerName="manager" Jul 21 12:23:44.292580 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.292531 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" containerName="manager" Jul 21 12:23:44.295684 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.295669 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:44.306192 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.306170 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs"] Jul 21 12:23:44.308483 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.308404 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.310420 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.310396 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.330359 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.330321 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.391369 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.391329 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f8ef-286f-46d4-9516-77a0a04975d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdtss\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.391528 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.391459 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z4l2\" (UniqueName: \"kubernetes.io/projected/33d38dd6-fd73-4b64-a223-5c2b137241d3-kube-api-access-2z4l2\") pod \"limitador-operator-controller-manager-859d7b78bd-ljkqs\" (UID: \"33d38dd6-fd73-4b64-a223-5c2b137241d3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:44.391528 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.391497 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdwjh\" (UniqueName: \"kubernetes.io/projected/f3a5f8ef-286f-46d4-9516-77a0a04975d1-kube-api-access-gdwjh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdtss\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.391860 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.391832 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f8ef-286f-46d4-9516-77a0a04975d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdtss\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.402391 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.402357 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdwjh\" (UniqueName: \"kubernetes.io/projected/f3a5f8ef-286f-46d4-9516-77a0a04975d1-kube-api-access-gdwjh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xdtss\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.492098 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.492057 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z4l2\" (UniqueName: \"kubernetes.io/projected/33d38dd6-fd73-4b64-a223-5c2b137241d3-kube-api-access-2z4l2\") pod \"limitador-operator-controller-manager-859d7b78bd-ljkqs\" (UID: \"33d38dd6-fd73-4b64-a223-5c2b137241d3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:44.507112 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.507083 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z4l2\" (UniqueName: \"kubernetes.io/projected/33d38dd6-fd73-4b64-a223-5c2b137241d3-kube-api-access-2z4l2\") pod \"limitador-operator-controller-manager-859d7b78bd-ljkqs\" (UID: \"33d38dd6-fd73-4b64-a223-5c2b137241d3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:44.510830 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.510810 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.512867 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.512851 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.513170 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.513150 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.514986 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.514963 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.516813 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.516791 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.518573 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.518552 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.598859 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.598615 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lll9\" (UniqueName: \"kubernetes.io/projected/8668e119-47a0-4cd3-9604-54d4a490b155-kube-api-access-4lll9\") pod \"8668e119-47a0-4cd3-9604-54d4a490b155\" (UID: \"8668e119-47a0-4cd3-9604-54d4a490b155\") " Jul 21 12:23:44.598859 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.598774 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057dc58c-b142-4264-8ddb-d4cb3d310aed-extensions-socket-volume\") pod \"057dc58c-b142-4264-8ddb-d4cb3d310aed\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " Jul 21 12:23:44.599094 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.598922 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l8vp\" (UniqueName: \"kubernetes.io/projected/057dc58c-b142-4264-8ddb-d4cb3d310aed-kube-api-access-2l8vp\") pod \"057dc58c-b142-4264-8ddb-d4cb3d310aed\" (UID: \"057dc58c-b142-4264-8ddb-d4cb3d310aed\") " Jul 21 12:23:44.599339 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.599310 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/057dc58c-b142-4264-8ddb-d4cb3d310aed-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "057dc58c-b142-4264-8ddb-d4cb3d310aed" (UID: "057dc58c-b142-4264-8ddb-d4cb3d310aed"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:23:44.599491 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.599416 2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/057dc58c-b142-4264-8ddb-d4cb3d310aed-extensions-socket-volume\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:44.600921 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.600897 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8668e119-47a0-4cd3-9604-54d4a490b155-kube-api-access-4lll9" (OuterVolumeSpecName: "kube-api-access-4lll9") pod "8668e119-47a0-4cd3-9604-54d4a490b155" (UID: "8668e119-47a0-4cd3-9604-54d4a490b155"). InnerVolumeSpecName "kube-api-access-4lll9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:23:44.601136 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.601107 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/057dc58c-b142-4264-8ddb-d4cb3d310aed-kube-api-access-2l8vp" (OuterVolumeSpecName: "kube-api-access-2l8vp") pod "057dc58c-b142-4264-8ddb-d4cb3d310aed" (UID: "057dc58c-b142-4264-8ddb-d4cb3d310aed"). InnerVolumeSpecName "kube-api-access-2l8vp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:23:44.655283 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.655246 2550 generic.go:356] "Generic (PLEG): container finished" podID="057dc58c-b142-4264-8ddb-d4cb3d310aed" containerID="00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8" exitCode=0 Jul 21 12:23:44.655484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.655311 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.655484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.655349 2550 scope.go:117] "RemoveContainer" containerID="00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8" Jul 21 12:23:44.656612 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.656584 2550 generic.go:356] "Generic (PLEG): container finished" podID="8668e119-47a0-4cd3-9604-54d4a490b155" containerID="d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65" exitCode=0 Jul 21 12:23:44.656725 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.656641 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.657739 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.657711 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.658293 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.658268 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:44.659816 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.659790 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.661859 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.661829 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.663681 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.663657 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.664744 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.664729 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:44.665970 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.665954 2550 scope.go:117] "RemoveContainer" containerID="00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8" Jul 21 12:23:44.666294 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.666276 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8\": container with ID starting with 00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8 not found: ID does not exist" containerID="00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8" Jul 21 12:23:44.666346 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.666303 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8"} err="failed to get container status \"00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8\": rpc error: code = NotFound desc = could not find container \"00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8\": container with ID starting with 00ce6fe7525c3a2e5948e58aec630b5af26a6cc22e65f15d298735847e8468d8 not found: ID does not exist" Jul 21 12:23:44.666346 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.666320 2550 scope.go:117] "RemoveContainer" containerID="d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65" Jul 21 12:23:44.670121 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.670096 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.672310 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.672282 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.674391 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.674364 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jjpd7\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jjpd7" Jul 21 12:23:44.679312 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.679283 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bbgkx\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbgkx" Jul 21 12:23:44.680316 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.680296 2550 scope.go:117] "RemoveContainer" containerID="d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65" Jul 21 12:23:44.680629 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:44.680607 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65\": container with ID starting with d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65 not found: ID does not exist" containerID="d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65" Jul 21 12:23:44.680711 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.680634 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65"} err="failed to get container status \"d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65\": rpc error: code = NotFound desc = could not find container \"d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65\": container with ID starting with d1aec38985bb45da5b828b030f711dd405c0a9a1b1e2d7fc91c886fb3cbf9d65 not found: ID does not exist" Jul 21 12:23:44.700216 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.700190 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lll9\" (UniqueName: \"kubernetes.io/projected/8668e119-47a0-4cd3-9604-54d4a490b155-kube-api-access-4lll9\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:44.700216 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.700215 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l8vp\" (UniqueName: \"kubernetes.io/projected/057dc58c-b142-4264-8ddb-d4cb3d310aed-kube-api-access-2l8vp\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:44.820849 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.820822 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss"] Jul 21 12:23:44.822330 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:44.822292 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a5f8ef_286f_46d4_9516_77a0a04975d1.slice/crio-ae9ee4ae4e84ae4808f2d36f155730963bf6b085911afa3771f47a5309de697d WatchSource:0}: Error finding container ae9ee4ae4e84ae4808f2d36f155730963bf6b085911afa3771f47a5309de697d: Status 404 returned error can't find the container with id ae9ee4ae4e84ae4808f2d36f155730963bf6b085911afa3771f47a5309de697d Jul 21 12:23:44.850962 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:44.850935 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs"] Jul 21 12:23:44.852458 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:23:44.852410 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d38dd6_fd73_4b64_a223_5c2b137241d3.slice/crio-577ef836b72689e19265990704b49905fdd1d499cf3db5593c1af4b67c93f249 WatchSource:0}: Error finding container 577ef836b72689e19265990704b49905fdd1d499cf3db5593c1af4b67c93f249: Status 404 returned error can't find the container with id 577ef836b72689e19265990704b49905fdd1d499cf3db5593c1af4b67c93f249 Jul 21 12:23:45.120937 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.120845 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057dc58c-b142-4264-8ddb-d4cb3d310aed" path="/var/lib/kubelet/pods/057dc58c-b142-4264-8ddb-d4cb3d310aed/volumes" Jul 21 12:23:45.121255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.121236 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8668e119-47a0-4cd3-9604-54d4a490b155" path="/var/lib/kubelet/pods/8668e119-47a0-4cd3-9604-54d4a490b155/volumes" Jul 21 12:23:45.607594 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.607563 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z774d" Jul 21 12:23:45.663513 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.663481 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" event={"ID":"f3a5f8ef-286f-46d4-9516-77a0a04975d1","Type":"ContainerStarted","Data":"195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e"} Jul 21 12:23:45.663695 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.663547 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:45.663695 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.663572 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" event={"ID":"f3a5f8ef-286f-46d4-9516-77a0a04975d1","Type":"ContainerStarted","Data":"ae9ee4ae4e84ae4808f2d36f155730963bf6b085911afa3771f47a5309de697d"} Jul 21 12:23:45.666844 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.666816 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" event={"ID":"33d38dd6-fd73-4b64-a223-5c2b137241d3","Type":"ContainerStarted","Data":"43404f51d6bea2f7cc92ec43721b9d21d15c0f17a87db90a324fdda5840ff663"} Jul 21 12:23:45.666963 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.666865 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" event={"ID":"33d38dd6-fd73-4b64-a223-5c2b137241d3","Type":"ContainerStarted","Data":"577ef836b72689e19265990704b49905fdd1d499cf3db5593c1af4b67c93f249"} Jul 21 12:23:45.666963 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.666892 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:45.707995 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.707940 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" podStartSLOduration=1.7079257650000002 podStartE2EDuration="1.707925765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:45.707338843 +0000 UTC m=+759.201167836" watchObservedRunningTime="2026-07-21 12:23:45.707925765 +0000 UTC m=+759.201754758" Jul 21 12:23:45.732847 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:45.732794 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" podStartSLOduration=1.732776815 podStartE2EDuration="1.732776815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:23:45.732557242 +0000 UTC m=+759.226386247" watchObservedRunningTime="2026-07-21 12:23:45.732776815 +0000 UTC m=+759.226605807" Jul 21 12:23:55.667766 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:55.667706 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cf7b8cdd6-swqnk" podUID="1bfd305a-a4cd-4002-a389-249430b87ab1" containerName="console" containerID="cri-o://e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5" gracePeriod=15 Jul 21 12:23:55.929919 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:55.929889 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:23:56.001524 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001495 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-serving-cert\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.001715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001554 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-oauth-config\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.001715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001590 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-service-ca\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.001715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001617 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5ldd\" (UniqueName: \"kubernetes.io/projected/1bfd305a-a4cd-4002-a389-249430b87ab1-kube-api-access-m5ldd\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.001715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001643 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-trusted-ca-bundle\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.001715 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001691 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-oauth-serving-cert\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.001940 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.001767 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-console-config\") pod \"1bfd305a-a4cd-4002-a389-249430b87ab1\" (UID: \"1bfd305a-a4cd-4002-a389-249430b87ab1\") " Jul 21 12:23:56.002099 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.002068 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-service-ca" (OuterVolumeSpecName: "service-ca") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:23:56.002233 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.002201 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:23:56.002442 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.002400 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-console-config" (OuterVolumeSpecName: "console-config") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:23:56.002512 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.002459 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:23:56.003940 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.003919 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bfd305a-a4cd-4002-a389-249430b87ab1-kube-api-access-m5ldd" (OuterVolumeSpecName: "kube-api-access-m5ldd") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "kube-api-access-m5ldd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:23:56.004217 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.004198 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:23:56.004264 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.004219 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1bfd305a-a4cd-4002-a389-249430b87ab1" (UID: "1bfd305a-a4cd-4002-a389-249430b87ab1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:23:56.103548 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103513 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-console-config\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.103548 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103544 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-serving-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.103548 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103556 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bfd305a-a4cd-4002-a389-249430b87ab1-console-oauth-config\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.103784 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103564 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.103784 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103573 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5ldd\" (UniqueName: \"kubernetes.io/projected/1bfd305a-a4cd-4002-a389-249430b87ab1-kube-api-access-m5ldd\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.103784 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103581 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-trusted-ca-bundle\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.103784 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.103590 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bfd305a-a4cd-4002-a389-249430b87ab1-oauth-serving-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:23:56.674934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.674904 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:23:56.675393 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.675102 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ljkqs" Jul 21 12:23:56.727612 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.727566 2550 generic.go:356] "Generic (PLEG): container finished" podID="1bfd305a-a4cd-4002-a389-249430b87ab1" containerID="e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5" exitCode=0 Jul 21 12:23:56.727802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.727622 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf7b8cdd6-swqnk" Jul 21 12:23:56.727802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.727654 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf7b8cdd6-swqnk" event={"ID":"1bfd305a-a4cd-4002-a389-249430b87ab1","Type":"ContainerDied","Data":"e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5"} Jul 21 12:23:56.727802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.727697 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf7b8cdd6-swqnk" event={"ID":"1bfd305a-a4cd-4002-a389-249430b87ab1","Type":"ContainerDied","Data":"8bc4687bdf567178291cc0ed8fce7d69a7477cbed94aa3d439666d2cc8dd72ce"} Jul 21 12:23:56.727802 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.727721 2550 scope.go:117] "RemoveContainer" containerID="e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5" Jul 21 12:23:56.739935 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.739905 2550 scope.go:117] "RemoveContainer" containerID="e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5" Jul 21 12:23:56.740234 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:23:56.740206 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5\": container with ID starting with e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5 not found: ID does not exist" containerID="e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5" Jul 21 12:23:56.740295 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.740245 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5"} err="failed to get container status \"e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5\": rpc error: code = NotFound desc = could not find container \"e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5\": container with ID starting with e9774c150f842b68ca6b38cdb4e03147efd1f4899d86a46584acd087528ea6e5 not found: ID does not exist" Jul 21 12:23:56.752853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.752825 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf7b8cdd6-swqnk"] Jul 21 12:23:56.754660 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:56.754636 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cf7b8cdd6-swqnk"] Jul 21 12:23:57.120744 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:23:57.120655 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bfd305a-a4cd-4002-a389-249430b87ab1" path="/var/lib/kubelet/pods/1bfd305a-a4cd-4002-a389-249430b87ab1/volumes" Jul 21 12:24:14.469252 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.469214 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss"] Jul 21 12:24:14.469776 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.469474 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" podUID="f3a5f8ef-286f-46d4-9516-77a0a04975d1" containerName="manager" containerID="cri-o://195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e" gracePeriod=10 Jul 21 12:24:14.714224 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.714192 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:24:14.825385 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.825298 2550 generic.go:356] "Generic (PLEG): container finished" podID="f3a5f8ef-286f-46d4-9516-77a0a04975d1" containerID="195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e" exitCode=0 Jul 21 12:24:14.825385 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.825345 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" Jul 21 12:24:14.825385 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.825370 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" event={"ID":"f3a5f8ef-286f-46d4-9516-77a0a04975d1","Type":"ContainerDied","Data":"195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e"} Jul 21 12:24:14.825691 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.825420 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss" event={"ID":"f3a5f8ef-286f-46d4-9516-77a0a04975d1","Type":"ContainerDied","Data":"ae9ee4ae4e84ae4808f2d36f155730963bf6b085911afa3771f47a5309de697d"} Jul 21 12:24:14.825691 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.825466 2550 scope.go:117] "RemoveContainer" containerID="195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e" Jul 21 12:24:14.834205 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.834188 2550 scope.go:117] "RemoveContainer" containerID="195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e" Jul 21 12:24:14.834475 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:24:14.834456 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e\": container with ID starting with 195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e not found: ID does not exist" containerID="195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e" Jul 21 12:24:14.834555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.834487 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e"} err="failed to get container status \"195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e\": rpc error: code = NotFound desc = could not find container \"195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e\": container with ID starting with 195ab75a362ddca541cc1c35fc378ed52c208127515254ade3e8e6ab2063307e not found: ID does not exist" Jul 21 12:24:14.876957 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.876923 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f8ef-286f-46d4-9516-77a0a04975d1-extensions-socket-volume\") pod \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " Jul 21 12:24:14.877144 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.877039 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdwjh\" (UniqueName: \"kubernetes.io/projected/f3a5f8ef-286f-46d4-9516-77a0a04975d1-kube-api-access-gdwjh\") pod \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\" (UID: \"f3a5f8ef-286f-46d4-9516-77a0a04975d1\") " Jul 21 12:24:14.877368 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.877340 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a5f8ef-286f-46d4-9516-77a0a04975d1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f3a5f8ef-286f-46d4-9516-77a0a04975d1" (UID: "f3a5f8ef-286f-46d4-9516-77a0a04975d1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:24:14.879102 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.879080 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a5f8ef-286f-46d4-9516-77a0a04975d1-kube-api-access-gdwjh" (OuterVolumeSpecName: "kube-api-access-gdwjh") pod "f3a5f8ef-286f-46d4-9516-77a0a04975d1" (UID: "f3a5f8ef-286f-46d4-9516-77a0a04975d1"). InnerVolumeSpecName "kube-api-access-gdwjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:24:14.978619 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.978582 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdwjh\" (UniqueName: \"kubernetes.io/projected/f3a5f8ef-286f-46d4-9516-77a0a04975d1-kube-api-access-gdwjh\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:24:14.978619 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:14.978617 2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f8ef-286f-46d4-9516-77a0a04975d1-extensions-socket-volume\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:24:15.145382 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:15.145349 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss"] Jul 21 12:24:15.151838 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:15.151815 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xdtss"] Jul 21 12:24:17.121244 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:17.121212 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a5f8ef-286f-46d4-9516-77a0a04975d1" path="/var/lib/kubelet/pods/f3a5f8ef-286f-46d4-9516-77a0a04975d1/volumes" Jul 21 12:24:30.722499 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.722406 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d"] Jul 21 12:24:30.722906 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.722889 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a5f8ef-286f-46d4-9516-77a0a04975d1" containerName="manager" Jul 21 12:24:30.722948 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.722908 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a5f8ef-286f-46d4-9516-77a0a04975d1" containerName="manager" Jul 21 12:24:30.722948 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.722925 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bfd305a-a4cd-4002-a389-249430b87ab1" containerName="console" Jul 21 12:24:30.722948 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.722930 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bfd305a-a4cd-4002-a389-249430b87ab1" containerName="console" Jul 21 12:24:30.723037 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.722995 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a5f8ef-286f-46d4-9516-77a0a04975d1" containerName="manager" Jul 21 12:24:30.723037 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.723005 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bfd305a-a4cd-4002-a389-249430b87ab1" containerName="console" Jul 21 12:24:30.728067 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.728045 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.730851 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.730830 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dqbb7\"" Jul 21 12:24:30.739315 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.739286 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d"] Jul 21 12:24:30.817873 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.817840 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.817873 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.817870 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cff849f-9b57-4685-85a2-6b0a707a6670-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.817896 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.817918 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.817971 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.818036 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818120 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.818085 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818336 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.818119 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjzzg\" (UniqueName: \"kubernetes.io/projected/9cff849f-9b57-4685-85a2-6b0a707a6670-kube-api-access-xjzzg\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.818336 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.818205 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919368 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919337 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919368 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919375 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjzzg\" (UniqueName: \"kubernetes.io/projected/9cff849f-9b57-4685-85a2-6b0a707a6670-kube-api-access-xjzzg\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919436 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919480 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919498 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cff849f-9b57-4685-85a2-6b0a707a6670-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919585 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919630 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919665 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919895 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.919975 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.919958 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.920137 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.920049 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.920137 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.920123 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.920261 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.920235 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cff849f-9b57-4685-85a2-6b0a707a6670-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.921964 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.921944 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.921964 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.921956 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.929927 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.929905 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cff849f-9b57-4685-85a2-6b0a707a6670-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:30.930059 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:30.930025 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjzzg\" (UniqueName: \"kubernetes.io/projected/9cff849f-9b57-4685-85a2-6b0a707a6670-kube-api-access-xjzzg\") pod \"maas-default-gateway-openshift-default-56654dcf7-glp4d\" (UID: \"9cff849f-9b57-4685-85a2-6b0a707a6670\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:31.039756 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.039659 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:31.188704 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.188678 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d"] Jul 21 12:24:31.190574 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:24:31.190545 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cff849f_9b57_4685_85a2_6b0a707a6670.slice/crio-af19de98f498535b566ece7ee61163c8cb7eff8ee5417cfd701c22269fa2dd68 WatchSource:0}: Error finding container af19de98f498535b566ece7ee61163c8cb7eff8ee5417cfd701c22269fa2dd68: Status 404 returned error can't find the container with id af19de98f498535b566ece7ee61163c8cb7eff8ee5417cfd701c22269fa2dd68 Jul 21 12:24:31.192725 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.192687 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:24:31.192811 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.192758 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:24:31.192811 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.192787 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:24:31.914615 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.914583 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" event={"ID":"9cff849f-9b57-4685-85a2-6b0a707a6670","Type":"ContainerStarted","Data":"e29d7ae5938f063223c53390a93473b8c35562a3e5bedd6d9bd2ec52303120a6"} Jul 21 12:24:31.914992 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.914636 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" event={"ID":"9cff849f-9b57-4685-85a2-6b0a707a6670","Type":"ContainerStarted","Data":"af19de98f498535b566ece7ee61163c8cb7eff8ee5417cfd701c22269fa2dd68"} Jul 21 12:24:31.936529 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:31.936479 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" podStartSLOduration=1.9364622919999999 podStartE2EDuration="1.936462292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:24:31.93502367 +0000 UTC m=+805.428852662" watchObservedRunningTime="2026-07-21 12:24:31.936462292 +0000 UTC m=+805.430291286" Jul 21 12:24:32.040492 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.040461 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:32.050113 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.050082 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:32.897219 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.897183 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:32.900933 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.900912 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:32.903534 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.903503 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 21 12:24:32.903644 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.903552 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mslb2\"" Jul 21 12:24:32.911039 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.911018 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:32.921173 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.921152 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:32.922215 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.922197 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:24:32.939517 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.939482 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4w8q\" (UniqueName: \"kubernetes.io/projected/7bc196a4-3829-43fa-bfe4-34fbaf603420-kube-api-access-g4w8q\") pod \"limitador-limitador-64c94fd64f-8tbx2\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:32.939623 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.939548 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc196a4-3829-43fa-bfe4-34fbaf603420-config-file\") pod \"limitador-limitador-64c94fd64f-8tbx2\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:32.995610 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:32.995576 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:33.040987 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.040951 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc196a4-3829-43fa-bfe4-34fbaf603420-config-file\") pod \"limitador-limitador-64c94fd64f-8tbx2\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:33.041220 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.041196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4w8q\" (UniqueName: \"kubernetes.io/projected/7bc196a4-3829-43fa-bfe4-34fbaf603420-kube-api-access-g4w8q\") pod \"limitador-limitador-64c94fd64f-8tbx2\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:33.041640 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.041619 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc196a4-3829-43fa-bfe4-34fbaf603420-config-file\") pod \"limitador-limitador-64c94fd64f-8tbx2\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:33.049416 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.049381 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4w8q\" (UniqueName: \"kubernetes.io/projected/7bc196a4-3829-43fa-bfe4-34fbaf603420-kube-api-access-g4w8q\") pod \"limitador-limitador-64c94fd64f-8tbx2\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:33.211902 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.211875 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:33.352303 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.352278 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:33.354338 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:24:33.354306 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc196a4_3829_43fa_bfe4_34fbaf603420.slice/crio-8f070a6e1123e781f2a2065b4d8bccb07c2dacbd552a49e74ccc85f304291c04 WatchSource:0}: Error finding container 8f070a6e1123e781f2a2065b4d8bccb07c2dacbd552a49e74ccc85f304291c04: Status 404 returned error can't find the container with id 8f070a6e1123e781f2a2065b4d8bccb07c2dacbd552a49e74ccc85f304291c04 Jul 21 12:24:33.685849 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.685820 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7scjc"] Jul 21 12:24:33.690634 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.690612 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:33.693038 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.693021 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jrdqk\"" Jul 21 12:24:33.701948 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.701109 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7scjc"] Jul 21 12:24:33.746189 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.746152 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6djfp\" (UniqueName: \"kubernetes.io/projected/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e-kube-api-access-6djfp\") pod \"authorino-58cd955675-7scjc\" (UID: \"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e\") " pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:33.847275 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.847243 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6djfp\" (UniqueName: \"kubernetes.io/projected/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e-kube-api-access-6djfp\") pod \"authorino-58cd955675-7scjc\" (UID: \"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e\") " pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:33.855999 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.855968 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6djfp\" (UniqueName: \"kubernetes.io/projected/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e-kube-api-access-6djfp\") pod \"authorino-58cd955675-7scjc\" (UID: \"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e\") " pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:33.926711 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:33.926678 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" event={"ID":"7bc196a4-3829-43fa-bfe4-34fbaf603420","Type":"ContainerStarted","Data":"8f070a6e1123e781f2a2065b4d8bccb07c2dacbd552a49e74ccc85f304291c04"} Jul 21 12:24:34.009640 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:34.009556 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:34.163776 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:34.163746 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7scjc"] Jul 21 12:24:34.166647 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:24:34.166616 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5ec6436_2fa9_48d6_b3f0_dfc24b9c923e.slice/crio-bff4a2a50841acee986f405478df787228a1e34971c093575b7aaa2c5d3107eb WatchSource:0}: Error finding container bff4a2a50841acee986f405478df787228a1e34971c093575b7aaa2c5d3107eb: Status 404 returned error can't find the container with id bff4a2a50841acee986f405478df787228a1e34971c093575b7aaa2c5d3107eb Jul 21 12:24:34.939628 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:34.938623 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7scjc" event={"ID":"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e","Type":"ContainerStarted","Data":"bff4a2a50841acee986f405478df787228a1e34971c093575b7aaa2c5d3107eb"} Jul 21 12:24:37.587507 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:37.587469 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7scjc"] Jul 21 12:24:38.965044 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:38.965011 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" event={"ID":"7bc196a4-3829-43fa-bfe4-34fbaf603420","Type":"ContainerStarted","Data":"03e27ff3dbf8e42649b373ae4e11d1f50cf93fd4392062dbacc6cf559fe22264"} Jul 21 12:24:38.965565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:38.965117 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:38.966544 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:38.966516 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7scjc" event={"ID":"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e","Type":"ContainerStarted","Data":"bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df"} Jul 21 12:24:38.966666 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:38.966574 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7scjc" podUID="c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" containerName="authorino" containerID="cri-o://bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df" gracePeriod=30 Jul 21 12:24:38.984987 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:38.984935 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" podStartSLOduration=2.258484296 podStartE2EDuration="6.984920221s" totalImagesPullingTime="4.726435925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:24:38.983880548 +0000 UTC m=+812.477709544" watchObservedRunningTime="2026-07-21 12:24:38.984920221 +0000 UTC m=+812.478749213" Jul 21 12:24:39.008342 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.008289 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7scjc" podStartSLOduration=2.087127309 podStartE2EDuration="6.008272459s" totalImagesPullingTime="3.92114515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:24:39.007812542 +0000 UTC m=+812.501641561" watchObservedRunningTime="2026-07-21 12:24:39.008272459 +0000 UTC m=+812.502101451" Jul 21 12:24:39.224945 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.224880 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:39.305344 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.305312 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6djfp\" (UniqueName: \"kubernetes.io/projected/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e-kube-api-access-6djfp\") pod \"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e\" (UID: \"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e\") " Jul 21 12:24:39.307268 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.307243 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e-kube-api-access-6djfp" (OuterVolumeSpecName: "kube-api-access-6djfp") pod "c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" (UID: "c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e"). InnerVolumeSpecName "kube-api-access-6djfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:24:39.406940 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.406905 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6djfp\" (UniqueName: \"kubernetes.io/projected/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e-kube-api-access-6djfp\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:24:39.973393 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.973362 2550 generic.go:356] "Generic (PLEG): container finished" podID="c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" containerID="bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df" exitCode=0 Jul 21 12:24:39.973904 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.973403 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7scjc" Jul 21 12:24:39.973904 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.973452 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7scjc" event={"ID":"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e","Type":"ContainerDied","Data":"bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df"} Jul 21 12:24:39.973904 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.973494 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7scjc" event={"ID":"c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e","Type":"ContainerDied","Data":"bff4a2a50841acee986f405478df787228a1e34971c093575b7aaa2c5d3107eb"} Jul 21 12:24:39.973904 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.973518 2550 scope.go:117] "RemoveContainer" containerID="bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df" Jul 21 12:24:39.982866 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.982848 2550 scope.go:117] "RemoveContainer" containerID="bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df" Jul 21 12:24:39.983097 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:24:39.983079 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df\": container with ID starting with bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df not found: ID does not exist" containerID="bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df" Jul 21 12:24:39.983157 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.983110 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df"} err="failed to get container status \"bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df\": rpc error: code = NotFound desc = could not find container \"bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df\": container with ID starting with bcd9664b463e212c57989b433d35b9ea21974ab2fe57897ca9aa8d5c23f897df not found: ID does not exist" Jul 21 12:24:39.996387 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:39.996358 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7scjc"] Jul 21 12:24:40.001484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:40.001461 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7scjc"] Jul 21 12:24:41.120551 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:41.120519 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" path="/var/lib/kubelet/pods/c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e/volumes" Jul 21 12:24:47.589801 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:47.589758 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:47.590252 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:47.590029 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" podUID="7bc196a4-3829-43fa-bfe4-34fbaf603420" containerName="limitador" containerID="cri-o://03e27ff3dbf8e42649b373ae4e11d1f50cf93fd4392062dbacc6cf559fe22264" gracePeriod=30 Jul 21 12:24:47.590890 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:47.590820 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:48.019273 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.019245 2550 generic.go:356] "Generic (PLEG): container finished" podID="7bc196a4-3829-43fa-bfe4-34fbaf603420" containerID="03e27ff3dbf8e42649b373ae4e11d1f50cf93fd4392062dbacc6cf559fe22264" exitCode=0 Jul 21 12:24:48.019417 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.019293 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" event={"ID":"7bc196a4-3829-43fa-bfe4-34fbaf603420","Type":"ContainerDied","Data":"03e27ff3dbf8e42649b373ae4e11d1f50cf93fd4392062dbacc6cf559fe22264"} Jul 21 12:24:48.146113 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.146083 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:48.290750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.290715 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4w8q\" (UniqueName: \"kubernetes.io/projected/7bc196a4-3829-43fa-bfe4-34fbaf603420-kube-api-access-g4w8q\") pod \"7bc196a4-3829-43fa-bfe4-34fbaf603420\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " Jul 21 12:24:48.290750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.290757 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc196a4-3829-43fa-bfe4-34fbaf603420-config-file\") pod \"7bc196a4-3829-43fa-bfe4-34fbaf603420\" (UID: \"7bc196a4-3829-43fa-bfe4-34fbaf603420\") " Jul 21 12:24:48.291111 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.291084 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bc196a4-3829-43fa-bfe4-34fbaf603420-config-file" (OuterVolumeSpecName: "config-file") pod "7bc196a4-3829-43fa-bfe4-34fbaf603420" (UID: "7bc196a4-3829-43fa-bfe4-34fbaf603420"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:24:48.292766 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.292746 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc196a4-3829-43fa-bfe4-34fbaf603420-kube-api-access-g4w8q" (OuterVolumeSpecName: "kube-api-access-g4w8q") pod "7bc196a4-3829-43fa-bfe4-34fbaf603420" (UID: "7bc196a4-3829-43fa-bfe4-34fbaf603420"). InnerVolumeSpecName "kube-api-access-g4w8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:24:48.392348 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.392306 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4w8q\" (UniqueName: \"kubernetes.io/projected/7bc196a4-3829-43fa-bfe4-34fbaf603420-kube-api-access-g4w8q\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:24:48.392348 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:48.392343 2550 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc196a4-3829-43fa-bfe4-34fbaf603420-config-file\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:24:49.027040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.027006 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" event={"ID":"7bc196a4-3829-43fa-bfe4-34fbaf603420","Type":"ContainerDied","Data":"8f070a6e1123e781f2a2065b4d8bccb07c2dacbd552a49e74ccc85f304291c04"} Jul 21 12:24:49.027040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.027040 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8tbx2" Jul 21 12:24:49.027594 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.027095 2550 scope.go:117] "RemoveContainer" containerID="03e27ff3dbf8e42649b373ae4e11d1f50cf93fd4392062dbacc6cf559fe22264" Jul 21 12:24:49.057444 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.057395 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:49.059995 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.059972 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8tbx2"] Jul 21 12:24:49.120241 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.120211 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bc196a4-3829-43fa-bfe4-34fbaf603420" path="/var/lib/kubelet/pods/7bc196a4-3829-43fa-bfe4-34fbaf603420/volumes" Jul 21 12:24:49.457710 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.457678 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn"] Jul 21 12:24:49.458146 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.458128 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" containerName="authorino" Jul 21 12:24:49.458255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.458148 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" containerName="authorino" Jul 21 12:24:49.458255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.458166 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bc196a4-3829-43fa-bfe4-34fbaf603420" containerName="limitador" Jul 21 12:24:49.458255 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.458175 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bc196a4-3829-43fa-bfe4-34fbaf603420" containerName="limitador" Jul 21 12:24:49.458409 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.458300 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bc196a4-3829-43fa-bfe4-34fbaf603420" containerName="limitador" Jul 21 12:24:49.458409 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.458317 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ec6436-2fa9-48d6-b3f0-dfc24b9c923e" containerName="authorino" Jul 21 12:24:49.462813 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.462793 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.465555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.465535 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 12:24:49.465678 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.465594 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 12:24:49.465752 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.465679 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4fhdk\"" Jul 21 12:24:49.465923 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.465905 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 21 12:24:49.470136 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.470115 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn"] Jul 21 12:24:49.601732 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.601697 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wddrj\" (UniqueName: \"kubernetes.io/projected/2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a-kube-api-access-wddrj\") pod \"postgres-69fb4594cf-hw7bn\" (UID: \"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.601911 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.601747 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a-data\") pod \"postgres-69fb4594cf-hw7bn\" (UID: \"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.702826 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.702788 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wddrj\" (UniqueName: \"kubernetes.io/projected/2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a-kube-api-access-wddrj\") pod \"postgres-69fb4594cf-hw7bn\" (UID: \"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.703016 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.702846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a-data\") pod \"postgres-69fb4594cf-hw7bn\" (UID: \"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.703239 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.703218 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a-data\") pod \"postgres-69fb4594cf-hw7bn\" (UID: \"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.712081 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.712014 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wddrj\" (UniqueName: \"kubernetes.io/projected/2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a-kube-api-access-wddrj\") pod \"postgres-69fb4594cf-hw7bn\" (UID: \"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.774783 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.774751 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:49.910751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:49.910720 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn"] Jul 21 12:24:49.913860 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:24:49.913836 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf943c3_7c52_42d0_98ae_ae4ce9a6e62a.slice/crio-8f11d06e415bb4fb5f3717297b76f8e185d92dba1cfd0dfd3295002a9c1420da WatchSource:0}: Error finding container 8f11d06e415bb4fb5f3717297b76f8e185d92dba1cfd0dfd3295002a9c1420da: Status 404 returned error can't find the container with id 8f11d06e415bb4fb5f3717297b76f8e185d92dba1cfd0dfd3295002a9c1420da Jul 21 12:24:50.034705 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:50.034611 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" event={"ID":"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a","Type":"ContainerStarted","Data":"8f11d06e415bb4fb5f3717297b76f8e185d92dba1cfd0dfd3295002a9c1420da"} Jul 21 12:24:55.071742 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:55.071647 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" event={"ID":"2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a","Type":"ContainerStarted","Data":"f408a5cae2723cb5d4a3a6d0879cb3db343edc374363fda07a4397f814532082"} Jul 21 12:24:55.072127 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:55.071742 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:24:55.089544 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:24:55.089497 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" podStartSLOduration=1.187891702 podStartE2EDuration="6.089481516s" totalImagesPullingTime="4.901589814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:24:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:24:55.088210491 +0000 UTC m=+828.582039484" watchObservedRunningTime="2026-07-21 12:24:55.089481516 +0000 UTC m=+828.583310509" Jul 21 12:25:01.108277 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.108250 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hw7bn" Jul 21 12:25:01.629376 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.629343 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lwvmb"] Jul 21 12:25:01.632982 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.632953 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:01.635785 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.635767 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jrdqk\"" Jul 21 12:25:01.641564 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.641539 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lwvmb"] Jul 21 12:25:01.829094 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.829055 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfqhf\" (UniqueName: \"kubernetes.io/projected/524849c1-8273-4def-a185-88047d0e179c-kube-api-access-dfqhf\") pod \"authorino-5d4b67c446-lwvmb\" (UID: \"524849c1-8273-4def-a185-88047d0e179c\") " pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:01.881061 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.880961 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lwvmb"] Jul 21 12:25:01.881342 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:25:01.881316 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-dfqhf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-lwvmb" podUID="524849c1-8273-4def-a185-88047d0e179c" Jul 21 12:25:01.916735 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.916696 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c6976df4d-vsnxk"] Jul 21 12:25:01.920440 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.920404 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:01.923070 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.923050 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 12:25:01.929762 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.929737 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfqhf\" (UniqueName: \"kubernetes.io/projected/524849c1-8273-4def-a185-88047d0e179c-kube-api-access-dfqhf\") pod \"authorino-5d4b67c446-lwvmb\" (UID: \"524849c1-8273-4def-a185-88047d0e179c\") " pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:01.930112 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.930092 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c6976df4d-vsnxk"] Jul 21 12:25:01.941603 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:01.941578 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfqhf\" (UniqueName: \"kubernetes.io/projected/524849c1-8273-4def-a185-88047d0e179c-kube-api-access-dfqhf\") pod \"authorino-5d4b67c446-lwvmb\" (UID: \"524849c1-8273-4def-a185-88047d0e179c\") " pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:02.031306 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.031270 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2cf23154-6395-43d0-8995-67761417cdbf-tls-cert\") pod \"authorino-6c6976df4d-vsnxk\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.031306 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.031303 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj2fw\" (UniqueName: \"kubernetes.io/projected/2cf23154-6395-43d0-8995-67761417cdbf-kube-api-access-jj2fw\") pod \"authorino-6c6976df4d-vsnxk\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.109294 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.109262 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:02.120396 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.120368 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:02.132363 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.132298 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2cf23154-6395-43d0-8995-67761417cdbf-tls-cert\") pod \"authorino-6c6976df4d-vsnxk\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.132363 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.132333 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj2fw\" (UniqueName: \"kubernetes.io/projected/2cf23154-6395-43d0-8995-67761417cdbf-kube-api-access-jj2fw\") pod \"authorino-6c6976df4d-vsnxk\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.134759 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.134738 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2cf23154-6395-43d0-8995-67761417cdbf-tls-cert\") pod \"authorino-6c6976df4d-vsnxk\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.140675 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.140654 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj2fw\" (UniqueName: \"kubernetes.io/projected/2cf23154-6395-43d0-8995-67761417cdbf-kube-api-access-jj2fw\") pod \"authorino-6c6976df4d-vsnxk\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.230851 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.230804 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:25:02.233204 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.233181 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfqhf\" (UniqueName: \"kubernetes.io/projected/524849c1-8273-4def-a185-88047d0e179c-kube-api-access-dfqhf\") pod \"524849c1-8273-4def-a185-88047d0e179c\" (UID: \"524849c1-8273-4def-a185-88047d0e179c\") " Jul 21 12:25:02.235546 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.235517 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/524849c1-8273-4def-a185-88047d0e179c-kube-api-access-dfqhf" (OuterVolumeSpecName: "kube-api-access-dfqhf") pod "524849c1-8273-4def-a185-88047d0e179c" (UID: "524849c1-8273-4def-a185-88047d0e179c"). InnerVolumeSpecName "kube-api-access-dfqhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:25:02.334085 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.334049 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfqhf\" (UniqueName: \"kubernetes.io/projected/524849c1-8273-4def-a185-88047d0e179c-kube-api-access-dfqhf\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:25:02.375488 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:02.375464 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c6976df4d-vsnxk"] Jul 21 12:25:02.377235 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:25:02.377209 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cf23154_6395_43d0_8995_67761417cdbf.slice/crio-ea83d3c639acaf13c7f8cd85aada68ad86d6bc38b2a4b5f807384efa277facbf WatchSource:0}: Error finding container ea83d3c639acaf13c7f8cd85aada68ad86d6bc38b2a4b5f807384efa277facbf: Status 404 returned error can't find the container with id ea83d3c639acaf13c7f8cd85aada68ad86d6bc38b2a4b5f807384efa277facbf Jul 21 12:25:03.118080 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:03.118049 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lwvmb" Jul 21 12:25:03.123409 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:03.123374 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" event={"ID":"2cf23154-6395-43d0-8995-67761417cdbf","Type":"ContainerStarted","Data":"b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2"} Jul 21 12:25:03.123606 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:03.123418 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" event={"ID":"2cf23154-6395-43d0-8995-67761417cdbf","Type":"ContainerStarted","Data":"ea83d3c639acaf13c7f8cd85aada68ad86d6bc38b2a4b5f807384efa277facbf"} Jul 21 12:25:03.134773 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:03.134724 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" podStartSLOduration=1.605310898 podStartE2EDuration="2.134708482s" totalImagesPullingTime="529.397584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:25:03.133918131 +0000 UTC m=+836.627747121" watchObservedRunningTime="2026-07-21 12:25:03.134708482 +0000 UTC m=+836.628537474" Jul 21 12:25:03.157404 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:03.157333 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lwvmb"] Jul 21 12:25:03.168295 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:03.168268 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lwvmb"] Jul 21 12:25:05.120828 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:05.120795 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="524849c1-8273-4def-a185-88047d0e179c" path="/var/lib/kubelet/pods/524849c1-8273-4def-a185-88047d0e179c/volumes" Jul 21 12:25:08.024029 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.023994 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7966fbf488-2rlpx"] Jul 21 12:25:08.047799 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.047768 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7966fbf488-2rlpx"] Jul 21 12:25:08.047950 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.047898 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.050529 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.050512 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hc8m7\"" Jul 21 12:25:08.050640 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.050511 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 21 12:25:08.051818 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.051796 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 21 12:25:08.084990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.084954 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ntsg\" (UniqueName: \"kubernetes.io/projected/932788a8-480b-4011-9b64-ea1b0d87e0c4-kube-api-access-8ntsg\") pod \"maas-controller-7966fbf488-2rlpx\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.085157 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.085000 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/932788a8-480b-4011-9b64-ea1b0d87e0c4-webhook-cert\") pod \"maas-controller-7966fbf488-2rlpx\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.185942 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.185909 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntsg\" (UniqueName: \"kubernetes.io/projected/932788a8-480b-4011-9b64-ea1b0d87e0c4-kube-api-access-8ntsg\") pod \"maas-controller-7966fbf488-2rlpx\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.186131 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.185959 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/932788a8-480b-4011-9b64-ea1b0d87e0c4-webhook-cert\") pod \"maas-controller-7966fbf488-2rlpx\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.188702 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.188674 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/932788a8-480b-4011-9b64-ea1b0d87e0c4-webhook-cert\") pod \"maas-controller-7966fbf488-2rlpx\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.194096 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.194074 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ntsg\" (UniqueName: \"kubernetes.io/projected/932788a8-480b-4011-9b64-ea1b0d87e0c4-kube-api-access-8ntsg\") pod \"maas-controller-7966fbf488-2rlpx\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.358674 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.358592 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:08.499529 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:08.499499 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7966fbf488-2rlpx"] Jul 21 12:25:08.503728 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:25:08.503705 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932788a8_480b_4011_9b64_ea1b0d87e0c4.slice/crio-cc26bc968826b4a2f49f2ac7287e285addb6c0820aa58bbc8dd7f1acce3edf8f WatchSource:0}: Error finding container cc26bc968826b4a2f49f2ac7287e285addb6c0820aa58bbc8dd7f1acce3edf8f: Status 404 returned error can't find the container with id cc26bc968826b4a2f49f2ac7287e285addb6c0820aa58bbc8dd7f1acce3edf8f Jul 21 12:25:09.154056 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:09.154017 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-2rlpx" event={"ID":"932788a8-480b-4011-9b64-ea1b0d87e0c4","Type":"ContainerStarted","Data":"cc26bc968826b4a2f49f2ac7287e285addb6c0820aa58bbc8dd7f1acce3edf8f"} Jul 21 12:25:11.166844 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:11.166757 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-2rlpx" event={"ID":"932788a8-480b-4011-9b64-ea1b0d87e0c4","Type":"ContainerStarted","Data":"6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387"} Jul 21 12:25:11.167246 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:11.166873 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:11.193341 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:11.193290 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7966fbf488-2rlpx" podStartSLOduration=0.821770677 podStartE2EDuration="3.193270325s" totalImagesPullingTime="2.371499648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:25:11.192122805 +0000 UTC m=+844.685951792" watchObservedRunningTime="2026-07-21 12:25:11.193270325 +0000 UTC m=+844.687099319" Jul 21 12:25:22.173650 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:22.173618 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:25:43.193532 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.193499 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb"] Jul 21 12:25:43.197561 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.197537 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.200184 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.200166 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 21 12:25:43.200277 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.200262 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzqbg\"" Jul 21 12:25:43.208042 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.208017 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb"] Jul 21 12:25:43.331364 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.331332 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-maas-api-tls\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.331573 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.331391 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcvzv\" (UniqueName: \"kubernetes.io/projected/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-kube-api-access-kcvzv\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.331573 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.331552 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-openshift-service-ca\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.432970 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.432935 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-maas-api-tls\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.433169 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.433006 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcvzv\" (UniqueName: \"kubernetes.io/projected/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-kube-api-access-kcvzv\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.433169 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.433052 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-openshift-service-ca\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.433782 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.433761 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-openshift-service-ca\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.435581 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.435557 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-maas-api-tls\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.441565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.441545 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcvzv\" (UniqueName: \"kubernetes.io/projected/9a9c2ffe-3280-452e-8426-14f02aa0cdc7-kube-api-access-kcvzv\") pod \"maas-api-74d896f5b9-hvctb\" (UID: \"9a9c2ffe-3280-452e-8426-14f02aa0cdc7\") " pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.509486 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.509379 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:43.665279 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:43.665249 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb"] Jul 21 12:25:43.666328 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:25:43.666293 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9c2ffe_3280_452e_8426_14f02aa0cdc7.slice/crio-aff58728be829883d3380e84d1fd14202de14ec671faa7ab743f1050607ce84d WatchSource:0}: Error finding container aff58728be829883d3380e84d1fd14202de14ec671faa7ab743f1050607ce84d: Status 404 returned error can't find the container with id aff58728be829883d3380e84d1fd14202de14ec671faa7ab743f1050607ce84d Jul 21 12:25:44.341108 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:44.341069 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" event={"ID":"9a9c2ffe-3280-452e-8426-14f02aa0cdc7","Type":"ContainerStarted","Data":"aff58728be829883d3380e84d1fd14202de14ec671faa7ab743f1050607ce84d"} Jul 21 12:25:46.365626 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:46.365590 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" event={"ID":"9a9c2ffe-3280-452e-8426-14f02aa0cdc7","Type":"ContainerStarted","Data":"81fe4ec150b5e5f31cce17dee045faa58ca7204280c39a041fdd70a07d7fc5a5"} Jul 21 12:25:46.366040 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:46.365738 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:46.384659 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:46.384608 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" podStartSLOduration=1.492034689 podStartE2EDuration="3.384595397s" totalImagesPullingTime="1.892560708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:25:46.383487391 +0000 UTC m=+879.877316382" watchObservedRunningTime="2026-07-21 12:25:46.384595397 +0000 UTC m=+879.878424390" Jul 21 12:25:52.376365 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.376336 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74d896f5b9-hvctb" Jul 21 12:25:52.406419 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.406383 2550 generic.go:356] "Generic (PLEG): container finished" podID="9cff849f-9b57-4685-85a2-6b0a707a6670" containerID="e29d7ae5938f063223c53390a93473b8c35562a3e5bedd6d9bd2ec52303120a6" exitCode=0 Jul 21 12:25:52.406635 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.406486 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" event={"ID":"9cff849f-9b57-4685-85a2-6b0a707a6670","Type":"ContainerDied","Data":"e29d7ae5938f063223c53390a93473b8c35562a3e5bedd6d9bd2ec52303120a6"} Jul 21 12:25:52.407278 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.406967 2550 scope.go:117] "RemoveContainer" containerID="e29d7ae5938f063223c53390a93473b8c35562a3e5bedd6d9bd2ec52303120a6" Jul 21 12:25:52.408409 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.408377 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:25:52.408504 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.408479 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:25:52.408552 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:52.408519 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 12:25:53.414202 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:53.414168 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" event={"ID":"9cff849f-9b57-4685-85a2-6b0a707a6670","Type":"ContainerStarted","Data":"dd99157907f0d8841e9f2f2b3fdb81dc32ad30442f3815ce67a52a0b0f442c28"} Jul 21 12:25:54.041202 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:54.041115 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:25:54.041546 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:54.041521 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-glp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 21 12:25:54.041739 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:54.041713 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" podUID="9cff849f-9b57-4685-85a2-6b0a707a6670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 21 12:25:55.040939 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:55.040902 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-glp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 21 12:25:55.041536 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:55.041503 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" podUID="9cff849f-9b57-4685-85a2-6b0a707a6670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 21 12:25:56.040105 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:56.040062 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-glp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 21 12:25:56.040310 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:56.040131 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" podUID="9cff849f-9b57-4685-85a2-6b0a707a6670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 21 12:25:57.040968 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:57.040924 2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-glp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 21 12:25:57.041588 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:57.041550 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" podUID="9cff849f-9b57-4685-85a2-6b0a707a6670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 21 12:25:58.043699 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:58.043666 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:25:58.044068 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:58.044017 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:25:58.044841 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:25:58.044826 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-glp4d" Jul 21 12:26:06.990750 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:06.990715 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:26:06.992167 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:06.992146 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:26:06.998870 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:06.998848 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:26:07.000274 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:07.000257 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:26:08.166222 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.166189 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr"] Jul 21 12:26:08.170068 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.170047 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.173902 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.173873 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 21 12:26:08.173902 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.173897 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 12:26:08.174069 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.173903 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-46ngs\"" Jul 21 12:26:08.174069 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.173904 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 12:26:08.178934 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.178913 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr"] Jul 21 12:26:08.288493 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.288461 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.288637 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.288504 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.288637 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.288527 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.288725 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.288644 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twvc9\" (UniqueName: \"kubernetes.io/projected/5a819487-9a20-4cc0-b836-745ef53b38b0-kube-api-access-twvc9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.288725 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.288680 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.288725 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.288716 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a819487-9a20-4cc0-b836-745ef53b38b0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.389844 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.389810 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.389993 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.389857 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a819487-9a20-4cc0-b836-745ef53b38b0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390047 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.389986 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390092 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.390052 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390092 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.390086 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390222 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.390201 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvc9\" (UniqueName: \"kubernetes.io/projected/5a819487-9a20-4cc0-b836-745ef53b38b0-kube-api-access-twvc9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390374 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.390352 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390490 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.390396 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.390550 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.390535 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.392134 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.392115 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a819487-9a20-4cc0-b836-745ef53b38b0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.392493 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.392476 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a819487-9a20-4cc0-b836-745ef53b38b0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.398045 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.398024 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twvc9\" (UniqueName: \"kubernetes.io/projected/5a819487-9a20-4cc0-b836-745ef53b38b0-kube-api-access-twvc9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr\" (UID: \"5a819487-9a20-4cc0-b836-745ef53b38b0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.482172 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.482119 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:08.633990 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.633961 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr"] Jul 21 12:26:08.637110 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:26:08.637073 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a819487_9a20_4cc0_b836_745ef53b38b0.slice/crio-f3c520e3a26757577a3d3f6670e8aa0eb1489e12700825f3345c772995658319 WatchSource:0}: Error finding container f3c520e3a26757577a3d3f6670e8aa0eb1489e12700825f3345c772995658319: Status 404 returned error can't find the container with id f3c520e3a26757577a3d3f6670e8aa0eb1489e12700825f3345c772995658319 Jul 21 12:26:08.639094 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:08.639078 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:26:09.508691 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:09.508659 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" event={"ID":"5a819487-9a20-4cc0-b836-745ef53b38b0","Type":"ContainerStarted","Data":"f3c520e3a26757577a3d3f6670e8aa0eb1489e12700825f3345c772995658319"} Jul 21 12:26:10.523712 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:10.523668 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" event={"ID":"5a819487-9a20-4cc0-b836-745ef53b38b0","Type":"ContainerStarted","Data":"7c0a6f2b63157e90c9d99ae581546cf952fd45145d72ad80a79300c7f5a00f4a"} Jul 21 12:26:10.544658 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:10.544601 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" podStartSLOduration=1.295966776 podStartE2EDuration="2.544580381s" totalImagesPullingTime="1.248613605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:26:10.540704727 +0000 UTC m=+904.034533724" watchObservedRunningTime="2026-07-21 12:26:10.544580381 +0000 UTC m=+904.038409374" Jul 21 12:26:18.482401 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:18.482368 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:18.482869 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:18.482454 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:18.495072 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:18.495044 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:18.581145 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:18.581119 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr" Jul 21 12:26:48.487103 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.487061 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l"] Jul 21 12:26:48.493751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.493727 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.496227 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.496205 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 21 12:26:48.504555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.504188 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l"] Jul 21 12:26:48.593998 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.593962 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.594170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.594010 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69n88\" (UniqueName: \"kubernetes.io/projected/646b691b-252a-4ee6-a6ad-50856937ba55-kube-api-access-69n88\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.594170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.594034 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/646b691b-252a-4ee6-a6ad-50856937ba55-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.594170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.594085 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.594170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.594114 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.594170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.594162 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.694668 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.694631 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.694861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.694718 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.694861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.694747 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69n88\" (UniqueName: \"kubernetes.io/projected/646b691b-252a-4ee6-a6ad-50856937ba55-kube-api-access-69n88\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.694861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.694768 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/646b691b-252a-4ee6-a6ad-50856937ba55-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.694861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.694790 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.694861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.694817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.695178 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.695154 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.695231 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.695186 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.695344 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.695326 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.697661 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.697610 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/646b691b-252a-4ee6-a6ad-50856937ba55-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.697661 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.697651 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/646b691b-252a-4ee6-a6ad-50856937ba55-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.708627 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.708602 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69n88\" (UniqueName: \"kubernetes.io/projected/646b691b-252a-4ee6-a6ad-50856937ba55-kube-api-access-69n88\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l\" (UID: \"646b691b-252a-4ee6-a6ad-50856937ba55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.809903 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.809810 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:48.955592 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:48.955563 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l"] Jul 21 12:26:48.957498 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:26:48.957467 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646b691b_252a_4ee6_a6ad_50856937ba55.slice/crio-ebd35df7da606fc4211f58992c5dc4f71bfd6408953db3822b52474b42ef3dd3 WatchSource:0}: Error finding container ebd35df7da606fc4211f58992c5dc4f71bfd6408953db3822b52474b42ef3dd3: Status 404 returned error can't find the container with id ebd35df7da606fc4211f58992c5dc4f71bfd6408953db3822b52474b42ef3dd3 Jul 21 12:26:49.741257 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:49.741218 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" event={"ID":"646b691b-252a-4ee6-a6ad-50856937ba55","Type":"ContainerStarted","Data":"a1b5366f1804dad98069412ef63c75a0eca12d04569d17542fc08a6ca0e5700f"} Jul 21 12:26:49.741687 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:49.741268 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" event={"ID":"646b691b-252a-4ee6-a6ad-50856937ba55","Type":"ContainerStarted","Data":"ebd35df7da606fc4211f58992c5dc4f71bfd6408953db3822b52474b42ef3dd3"} Jul 21 12:26:49.765840 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:49.765788 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" podStartSLOduration=1.541600151 podStartE2EDuration="1.765773445s" totalImagesPullingTime="224.173294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:26:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:26:49.764595815 +0000 UTC m=+943.258424809" watchObservedRunningTime="2026-07-21 12:26:49.765773445 +0000 UTC m=+943.259602443" Jul 21 12:26:58.810815 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:58.810781 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:58.811275 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:58.810850 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:58.823215 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:58.823189 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:26:59.812732 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:26:59.812701 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l" Jul 21 12:27:46.257759 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.257684 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75674d865c-cw8r6"] Jul 21 12:27:46.261652 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.261625 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.271731 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.271703 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75674d865c-cw8r6"] Jul 21 12:27:46.368107 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.368072 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f83c5620-4886-4384-a8e8-66aff32b36a8-tls-cert\") pod \"authorino-75674d865c-cw8r6\" (UID: \"f83c5620-4886-4384-a8e8-66aff32b36a8\") " pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.368301 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.368193 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlcm5\" (UniqueName: \"kubernetes.io/projected/f83c5620-4886-4384-a8e8-66aff32b36a8-kube-api-access-jlcm5\") pod \"authorino-75674d865c-cw8r6\" (UID: \"f83c5620-4886-4384-a8e8-66aff32b36a8\") " pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.469160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.469126 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f83c5620-4886-4384-a8e8-66aff32b36a8-tls-cert\") pod \"authorino-75674d865c-cw8r6\" (UID: \"f83c5620-4886-4384-a8e8-66aff32b36a8\") " pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.469339 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.469178 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlcm5\" (UniqueName: \"kubernetes.io/projected/f83c5620-4886-4384-a8e8-66aff32b36a8-kube-api-access-jlcm5\") pod \"authorino-75674d865c-cw8r6\" (UID: \"f83c5620-4886-4384-a8e8-66aff32b36a8\") " pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.471620 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.471597 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f83c5620-4886-4384-a8e8-66aff32b36a8-tls-cert\") pod \"authorino-75674d865c-cw8r6\" (UID: \"f83c5620-4886-4384-a8e8-66aff32b36a8\") " pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.477176 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.477149 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlcm5\" (UniqueName: \"kubernetes.io/projected/f83c5620-4886-4384-a8e8-66aff32b36a8-kube-api-access-jlcm5\") pod \"authorino-75674d865c-cw8r6\" (UID: \"f83c5620-4886-4384-a8e8-66aff32b36a8\") " pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.571694 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.571614 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75674d865c-cw8r6" Jul 21 12:27:46.718409 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:46.718383 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75674d865c-cw8r6"] Jul 21 12:27:46.719993 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:27:46.719968 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf83c5620_4886_4384_a8e8_66aff32b36a8.slice/crio-398472f1cc28d2ce8cf6b829483003f7c383d38a97b66e35ca1152e70db6b328 WatchSource:0}: Error finding container 398472f1cc28d2ce8cf6b829483003f7c383d38a97b66e35ca1152e70db6b328: Status 404 returned error can't find the container with id 398472f1cc28d2ce8cf6b829483003f7c383d38a97b66e35ca1152e70db6b328 Jul 21 12:27:47.058794 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:47.058761 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75674d865c-cw8r6" event={"ID":"f83c5620-4886-4384-a8e8-66aff32b36a8","Type":"ContainerStarted","Data":"398472f1cc28d2ce8cf6b829483003f7c383d38a97b66e35ca1152e70db6b328"} Jul 21 12:27:49.074294 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.074249 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75674d865c-cw8r6" event={"ID":"f83c5620-4886-4384-a8e8-66aff32b36a8","Type":"ContainerStarted","Data":"418905a15e0f12363bb873586bc716b387a7403fee69c6b6dedb4d646dd7bcd4"} Jul 21 12:27:49.098334 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.098269 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75674d865c-cw8r6" podStartSLOduration=1.82405634 podStartE2EDuration="3.098244383s" totalImagesPullingTime="1.274188043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:49.094791423 +0000 UTC m=+1002.588620420" watchObservedRunningTime="2026-07-21 12:27:49.098244383 +0000 UTC m=+1002.592073376" Jul 21 12:27:49.141122 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.141090 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c6976df4d-vsnxk"] Jul 21 12:27:49.141370 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.141273 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" podUID="2cf23154-6395-43d0-8995-67761417cdbf" containerName="authorino" containerID="cri-o://b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2" gracePeriod=30 Jul 21 12:27:49.406314 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.406288 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:27:49.501364 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.501206 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2cf23154-6395-43d0-8995-67761417cdbf-tls-cert\") pod \"2cf23154-6395-43d0-8995-67761417cdbf\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " Jul 21 12:27:49.502314 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.502290 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj2fw\" (UniqueName: \"kubernetes.io/projected/2cf23154-6395-43d0-8995-67761417cdbf-kube-api-access-jj2fw\") pod \"2cf23154-6395-43d0-8995-67761417cdbf\" (UID: \"2cf23154-6395-43d0-8995-67761417cdbf\") " Jul 21 12:27:49.507104 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.507054 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cf23154-6395-43d0-8995-67761417cdbf-kube-api-access-jj2fw" (OuterVolumeSpecName: "kube-api-access-jj2fw") pod "2cf23154-6395-43d0-8995-67761417cdbf" (UID: "2cf23154-6395-43d0-8995-67761417cdbf"). InnerVolumeSpecName "kube-api-access-jj2fw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:27:49.535612 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.535579 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cf23154-6395-43d0-8995-67761417cdbf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2cf23154-6395-43d0-8995-67761417cdbf" (UID: "2cf23154-6395-43d0-8995-67761417cdbf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:27:49.604008 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.603973 2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2cf23154-6395-43d0-8995-67761417cdbf-tls-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:27:49.604008 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:49.604002 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj2fw\" (UniqueName: \"kubernetes.io/projected/2cf23154-6395-43d0-8995-67761417cdbf-kube-api-access-jj2fw\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:27:50.081106 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.081070 2550 generic.go:356] "Generic (PLEG): container finished" podID="2cf23154-6395-43d0-8995-67761417cdbf" containerID="b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2" exitCode=0 Jul 21 12:27:50.081629 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.081120 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" Jul 21 12:27:50.081629 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.081161 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" event={"ID":"2cf23154-6395-43d0-8995-67761417cdbf","Type":"ContainerDied","Data":"b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2"} Jul 21 12:27:50.081629 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.081207 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6976df4d-vsnxk" event={"ID":"2cf23154-6395-43d0-8995-67761417cdbf","Type":"ContainerDied","Data":"ea83d3c639acaf13c7f8cd85aada68ad86d6bc38b2a4b5f807384efa277facbf"} Jul 21 12:27:50.081629 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.081225 2550 scope.go:117] "RemoveContainer" containerID="b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2" Jul 21 12:27:50.098851 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.098810 2550 scope.go:117] "RemoveContainer" containerID="b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2" Jul 21 12:27:50.099388 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:27:50.099334 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2\": container with ID starting with b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2 not found: ID does not exist" containerID="b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2" Jul 21 12:27:50.099485 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.099415 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2"} err="failed to get container status \"b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2\": rpc error: code = NotFound desc = could not find container \"b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2\": container with ID starting with b6165b886a3dd9b22b1391d1070818a572fbe4e705da3e922ff031396919ebc2 not found: ID does not exist" Jul 21 12:27:50.108471 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.108438 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c6976df4d-vsnxk"] Jul 21 12:27:50.110544 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:50.110521 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c6976df4d-vsnxk"] Jul 21 12:27:51.121250 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:27:51.121219 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cf23154-6395-43d0-8995-67761417cdbf" path="/var/lib/kubelet/pods/2cf23154-6395-43d0-8995-67761417cdbf/volumes" Jul 21 12:28:41.022209 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.022177 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7966fbf488-2rlpx"] Jul 21 12:28:41.022838 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.022481 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7966fbf488-2rlpx" podUID="932788a8-480b-4011-9b64-ea1b0d87e0c4" containerName="manager" containerID="cri-o://6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387" gracePeriod=30 Jul 21 12:28:41.287836 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.287769 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:28:41.331520 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.331477 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ntsg\" (UniqueName: \"kubernetes.io/projected/932788a8-480b-4011-9b64-ea1b0d87e0c4-kube-api-access-8ntsg\") pod \"932788a8-480b-4011-9b64-ea1b0d87e0c4\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " Jul 21 12:28:41.331706 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.331574 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/932788a8-480b-4011-9b64-ea1b0d87e0c4-webhook-cert\") pod \"932788a8-480b-4011-9b64-ea1b0d87e0c4\" (UID: \"932788a8-480b-4011-9b64-ea1b0d87e0c4\") " Jul 21 12:28:41.333813 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.333778 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/932788a8-480b-4011-9b64-ea1b0d87e0c4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "932788a8-480b-4011-9b64-ea1b0d87e0c4" (UID: "932788a8-480b-4011-9b64-ea1b0d87e0c4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:28:41.333813 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.333801 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932788a8-480b-4011-9b64-ea1b0d87e0c4-kube-api-access-8ntsg" (OuterVolumeSpecName: "kube-api-access-8ntsg") pod "932788a8-480b-4011-9b64-ea1b0d87e0c4" (UID: "932788a8-480b-4011-9b64-ea1b0d87e0c4"). InnerVolumeSpecName "kube-api-access-8ntsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:28:41.377765 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.377738 2550 generic.go:356] "Generic (PLEG): container finished" podID="932788a8-480b-4011-9b64-ea1b0d87e0c4" containerID="6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387" exitCode=0 Jul 21 12:28:41.377925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.377783 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7966fbf488-2rlpx" Jul 21 12:28:41.377925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.377823 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-2rlpx" event={"ID":"932788a8-480b-4011-9b64-ea1b0d87e0c4","Type":"ContainerDied","Data":"6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387"} Jul 21 12:28:41.377925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.377880 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7966fbf488-2rlpx" event={"ID":"932788a8-480b-4011-9b64-ea1b0d87e0c4","Type":"ContainerDied","Data":"cc26bc968826b4a2f49f2ac7287e285addb6c0820aa58bbc8dd7f1acce3edf8f"} Jul 21 12:28:41.377925 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.377910 2550 scope.go:117] "RemoveContainer" containerID="6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387" Jul 21 12:28:41.388015 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.387997 2550 scope.go:117] "RemoveContainer" containerID="6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387" Jul 21 12:28:41.388294 ip-10-0-136-33 kubenswrapper[2550]: E0721 12:28:41.388274 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387\": container with ID starting with 6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387 not found: ID does not exist" containerID="6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387" Jul 21 12:28:41.388344 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.388303 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387"} err="failed to get container status \"6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387\": rpc error: code = NotFound desc = could not find container \"6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387\": container with ID starting with 6419d393b247f9ea194b95715731a921217cd133d402594c95173c027f63a387 not found: ID does not exist" Jul 21 12:28:41.405268 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.405239 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7966fbf488-2rlpx"] Jul 21 12:28:41.419571 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.419542 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7966fbf488-2rlpx"] Jul 21 12:28:41.432648 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.432624 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ntsg\" (UniqueName: \"kubernetes.io/projected/932788a8-480b-4011-9b64-ea1b0d87e0c4-kube-api-access-8ntsg\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:28:41.432648 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:41.432648 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/932788a8-480b-4011-9b64-ea1b0d87e0c4-webhook-cert\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:28:43.121131 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:28:43.121094 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932788a8-480b-4011-9b64-ea1b0d87e0c4" path="/var/lib/kubelet/pods/932788a8-480b-4011-9b64-ea1b0d87e0c4/volumes" Jul 21 12:31:07.023972 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:31:07.023941 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:31:07.026587 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:31:07.026564 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:31:07.031647 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:31:07.031630 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:31:07.034322 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:31:07.034306 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:36:07.058053 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:36:07.058020 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:36:07.066053 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:36:07.066026 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:36:07.068456 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:36:07.068420 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:36:07.075473 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:36:07.075417 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:41:07.106771 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:41:07.106659 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:41:07.122121 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:41:07.122032 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:41:07.122924 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:41:07.122895 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:41:07.133853 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:41:07.133826 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:43:23.705565 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.705528 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv"] Jul 21 12:43:23.706163 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.706144 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932788a8-480b-4011-9b64-ea1b0d87e0c4" containerName="manager" Jul 21 12:43:23.706247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.706167 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="932788a8-480b-4011-9b64-ea1b0d87e0c4" containerName="manager" Jul 21 12:43:23.706247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.706196 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf23154-6395-43d0-8995-67761417cdbf" containerName="authorino" Jul 21 12:43:23.706247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.706205 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf23154-6395-43d0-8995-67761417cdbf" containerName="authorino" Jul 21 12:43:23.706402 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.706336 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cf23154-6395-43d0-8995-67761417cdbf" containerName="authorino" Jul 21 12:43:23.706402 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.706353 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="932788a8-480b-4011-9b64-ea1b0d87e0c4" containerName="manager" Jul 21 12:43:23.709602 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.709582 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:23.712314 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.712290 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:43:23.716861 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.716839 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv"] Jul 21 12:43:23.880140 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.880097 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb450280-0ef2-47e2-88af-057d2ee26e7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv\" (UID: \"eb450280-0ef2-47e2-88af-057d2ee26e7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:23.981597 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.981494 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb450280-0ef2-47e2-88af-057d2ee26e7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv\" (UID: \"eb450280-0ef2-47e2-88af-057d2ee26e7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:23.982117 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:23.982099 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb450280-0ef2-47e2-88af-057d2ee26e7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv\" (UID: \"eb450280-0ef2-47e2-88af-057d2ee26e7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:24.021685 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:24.021655 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:24.370858 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:24.370830 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv"] Jul 21 12:43:24.371472 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:43:24.371417 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb450280_0ef2_47e2_88af_057d2ee26e7c.slice/crio-70f690565701156a4489d02532e98f5329e6784c04208ae997bf04e4a4b413b1 WatchSource:0}: Error finding container 70f690565701156a4489d02532e98f5329e6784c04208ae997bf04e4a4b413b1: Status 404 returned error can't find the container with id 70f690565701156a4489d02532e98f5329e6784c04208ae997bf04e4a4b413b1 Jul 21 12:43:24.373388 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:24.373370 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:43:25.271738 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:25.271707 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" event={"ID":"eb450280-0ef2-47e2-88af-057d2ee26e7c","Type":"ContainerStarted","Data":"70f690565701156a4489d02532e98f5329e6784c04208ae997bf04e4a4b413b1"} Jul 21 12:43:27.285025 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:27.284926 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" event={"ID":"eb450280-0ef2-47e2-88af-057d2ee26e7c","Type":"ContainerStarted","Data":"ab9c84fd86d71281e57937833ebeeda25acf9eb3873b5b7e1f1f7e83c56eee92"} Jul 21 12:43:27.302199 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:27.302092 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" podStartSLOduration=2.020953769 podStartE2EDuration="4.302074528s" totalImagesPullingTime="2.281120759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:43:27.301718283 +0000 UTC m=+1940.795547306" watchObservedRunningTime="2026-07-21 12:43:27.302074528 +0000 UTC m=+1940.795903526" Jul 21 12:43:28.291858 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:28.291819 2550 generic.go:356] "Generic (PLEG): container finished" podID="eb450280-0ef2-47e2-88af-057d2ee26e7c" containerID="ab9c84fd86d71281e57937833ebeeda25acf9eb3873b5b7e1f1f7e83c56eee92" exitCode=0 Jul 21 12:43:28.292252 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:28.291893 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" event={"ID":"eb450280-0ef2-47e2-88af-057d2ee26e7c","Type":"ContainerDied","Data":"ab9c84fd86d71281e57937833ebeeda25acf9eb3873b5b7e1f1f7e83c56eee92"} Jul 21 12:43:29.443807 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:29.443784 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:29.535648 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:29.535610 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb450280-0ef2-47e2-88af-057d2ee26e7c-maas-api-service-ca\") pod \"eb450280-0ef2-47e2-88af-057d2ee26e7c\" (UID: \"eb450280-0ef2-47e2-88af-057d2ee26e7c\") " Jul 21 12:43:29.535940 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:29.535917 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb450280-0ef2-47e2-88af-057d2ee26e7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb450280-0ef2-47e2-88af-057d2ee26e7c" (UID: "eb450280-0ef2-47e2-88af-057d2ee26e7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:43:29.637178 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:29.637093 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb450280-0ef2-47e2-88af-057d2ee26e7c-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:43:30.304253 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:30.304215 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" event={"ID":"eb450280-0ef2-47e2-88af-057d2ee26e7c","Type":"ContainerDied","Data":"70f690565701156a4489d02532e98f5329e6784c04208ae997bf04e4a4b413b1"} Jul 21 12:43:30.304453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:30.304270 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f690565701156a4489d02532e98f5329e6784c04208ae997bf04e4a4b413b1" Jul 21 12:43:30.304453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:30.304238 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv" Jul 21 12:43:38.783879 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:38.783845 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv"] Jul 21 12:43:38.788160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:38.788133 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9fb423b2-tgchv"] Jul 21 12:43:39.125482 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:43:39.125367 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb450280-0ef2-47e2-88af-057d2ee26e7c" path="/var/lib/kubelet/pods/eb450280-0ef2-47e2-88af-057d2ee26e7c/volumes" Jul 21 12:44:07.525219 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.525181 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6"] Jul 21 12:44:07.525825 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.525805 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb450280-0ef2-47e2-88af-057d2ee26e7c" containerName="revoke-keys" Jul 21 12:44:07.525906 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.525827 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb450280-0ef2-47e2-88af-057d2ee26e7c" containerName="revoke-keys" Jul 21 12:44:07.525998 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.525986 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb450280-0ef2-47e2-88af-057d2ee26e7c" containerName="revoke-keys" Jul 21 12:44:07.533112 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.533088 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:07.535793 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.535770 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:44:07.535985 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.535960 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6"] Jul 21 12:44:07.702624 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.702590 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b648b0-dacb-41b8-9907-442189934be7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6\" (UID: \"14b648b0-dacb-41b8-9907-442189934be7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:07.804193 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.804100 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b648b0-dacb-41b8-9907-442189934be7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6\" (UID: \"14b648b0-dacb-41b8-9907-442189934be7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:07.804762 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.804741 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b648b0-dacb-41b8-9907-442189934be7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6\" (UID: \"14b648b0-dacb-41b8-9907-442189934be7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:07.845782 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.845750 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:07.991917 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:07.991889 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6"] Jul 21 12:44:07.993873 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:44:07.993841 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b648b0_dacb_41b8_9907_442189934be7.slice/crio-2348682bef769f233dcd95067bc8b3d68c1d8fc86cb307878d17bd2af76941e7 WatchSource:0}: Error finding container 2348682bef769f233dcd95067bc8b3d68c1d8fc86cb307878d17bd2af76941e7: Status 404 returned error can't find the container with id 2348682bef769f233dcd95067bc8b3d68c1d8fc86cb307878d17bd2af76941e7 Jul 21 12:44:08.525162 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:08.525073 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" event={"ID":"14b648b0-dacb-41b8-9907-442189934be7","Type":"ContainerStarted","Data":"8b827740ecd7541bafd54b003c8f6343ba721c22a0d1a4b0bccb180f1516e2f4"} Jul 21 12:44:08.525162 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:08.525122 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" event={"ID":"14b648b0-dacb-41b8-9907-442189934be7","Type":"ContainerStarted","Data":"2348682bef769f233dcd95067bc8b3d68c1d8fc86cb307878d17bd2af76941e7"} Jul 21 12:44:08.544906 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:08.544855 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" podStartSLOduration=1.5448388849999999 podStartE2EDuration="1.544838885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:44:08.543717467 +0000 UTC m=+1982.037546460" watchObservedRunningTime="2026-07-21 12:44:08.544838885 +0000 UTC m=+1982.038667877" Jul 21 12:44:09.531403 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:09.531368 2550 generic.go:356] "Generic (PLEG): container finished" podID="14b648b0-dacb-41b8-9907-442189934be7" containerID="8b827740ecd7541bafd54b003c8f6343ba721c22a0d1a4b0bccb180f1516e2f4" exitCode=0 Jul 21 12:44:09.531933 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:09.531473 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" event={"ID":"14b648b0-dacb-41b8-9907-442189934be7","Type":"ContainerDied","Data":"8b827740ecd7541bafd54b003c8f6343ba721c22a0d1a4b0bccb180f1516e2f4"} Jul 21 12:44:10.681844 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:10.681821 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:10.835351 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:10.835255 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b648b0-dacb-41b8-9907-442189934be7-maas-api-service-ca\") pod \"14b648b0-dacb-41b8-9907-442189934be7\" (UID: \"14b648b0-dacb-41b8-9907-442189934be7\") " Jul 21 12:44:10.835685 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:10.835659 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b648b0-dacb-41b8-9907-442189934be7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "14b648b0-dacb-41b8-9907-442189934be7" (UID: "14b648b0-dacb-41b8-9907-442189934be7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:44:10.935968 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:10.935938 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b648b0-dacb-41b8-9907-442189934be7-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:44:11.544583 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:11.544549 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" event={"ID":"14b648b0-dacb-41b8-9907-442189934be7","Type":"ContainerDied","Data":"2348682bef769f233dcd95067bc8b3d68c1d8fc86cb307878d17bd2af76941e7"} Jul 21 12:44:11.544583 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:11.544584 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6" Jul 21 12:44:11.544795 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:11.544596 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2348682bef769f233dcd95067bc8b3d68c1d8fc86cb307878d17bd2af76941e7" Jul 21 12:44:22.671038 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:22.671006 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6"] Jul 21 12:44:22.677725 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:22.677700 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-21bec010-mb9j6"] Jul 21 12:44:23.120877 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:23.120845 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b648b0-dacb-41b8-9907-442189934be7" path="/var/lib/kubelet/pods/14b648b0-dacb-41b8-9907-442189934be7/volumes" Jul 21 12:44:51.456670 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.456638 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq"] Jul 21 12:44:51.457137 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.457054 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b648b0-dacb-41b8-9907-442189934be7" containerName="revoke-keys" Jul 21 12:44:51.457137 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.457064 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b648b0-dacb-41b8-9907-442189934be7" containerName="revoke-keys" Jul 21 12:44:51.457210 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.457154 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b648b0-dacb-41b8-9907-442189934be7" containerName="revoke-keys" Jul 21 12:44:51.460362 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.460347 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:51.464724 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.464704 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:44:51.469524 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.469500 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq"] Jul 21 12:44:51.636852 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.636816 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68d0e17-7808-425f-b9d3-ea2d4419db86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq\" (UID: \"e68d0e17-7808-425f-b9d3-ea2d4419db86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:51.738387 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.738289 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68d0e17-7808-425f-b9d3-ea2d4419db86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq\" (UID: \"e68d0e17-7808-425f-b9d3-ea2d4419db86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:51.738987 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.738961 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68d0e17-7808-425f-b9d3-ea2d4419db86-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq\" (UID: \"e68d0e17-7808-425f-b9d3-ea2d4419db86\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:51.772439 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.772401 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:51.916316 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:51.916284 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq"] Jul 21 12:44:51.917537 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:44:51.917511 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68d0e17_7808_425f_b9d3_ea2d4419db86.slice/crio-25a3cae326d799bd97ac0eb57b1aa28630ea6ee25325abb732bb6f8d98ea3b6d WatchSource:0}: Error finding container 25a3cae326d799bd97ac0eb57b1aa28630ea6ee25325abb732bb6f8d98ea3b6d: Status 404 returned error can't find the container with id 25a3cae326d799bd97ac0eb57b1aa28630ea6ee25325abb732bb6f8d98ea3b6d Jul 21 12:44:52.777095 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:52.777051 2550 generic.go:356] "Generic (PLEG): container finished" podID="e68d0e17-7808-425f-b9d3-ea2d4419db86" containerID="5a957f8e0d2c1636a76041c3117c66aaba2e797ccaf929152e6e52955deb72ce" exitCode=0 Jul 21 12:44:52.777461 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:52.777135 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" event={"ID":"e68d0e17-7808-425f-b9d3-ea2d4419db86","Type":"ContainerDied","Data":"5a957f8e0d2c1636a76041c3117c66aaba2e797ccaf929152e6e52955deb72ce"} Jul 21 12:44:52.777461 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:52.777214 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" event={"ID":"e68d0e17-7808-425f-b9d3-ea2d4419db86","Type":"ContainerStarted","Data":"25a3cae326d799bd97ac0eb57b1aa28630ea6ee25325abb732bb6f8d98ea3b6d"} Jul 21 12:44:53.929276 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:53.929251 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:54.059247 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:54.059156 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68d0e17-7808-425f-b9d3-ea2d4419db86-maas-api-service-ca\") pod \"e68d0e17-7808-425f-b9d3-ea2d4419db86\" (UID: \"e68d0e17-7808-425f-b9d3-ea2d4419db86\") " Jul 21 12:44:54.059557 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:54.059534 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e68d0e17-7808-425f-b9d3-ea2d4419db86-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e68d0e17-7808-425f-b9d3-ea2d4419db86" (UID: "e68d0e17-7808-425f-b9d3-ea2d4419db86"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:44:54.159899 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:54.159866 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68d0e17-7808-425f-b9d3-ea2d4419db86-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:44:54.789682 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:54.789654 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" Jul 21 12:44:54.789885 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:54.789692 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq" event={"ID":"e68d0e17-7808-425f-b9d3-ea2d4419db86","Type":"ContainerDied","Data":"25a3cae326d799bd97ac0eb57b1aa28630ea6ee25325abb732bb6f8d98ea3b6d"} Jul 21 12:44:54.789885 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:44:54.789738 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25a3cae326d799bd97ac0eb57b1aa28630ea6ee25325abb732bb6f8d98ea3b6d" Jul 21 12:45:00.172235 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.172198 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl"] Jul 21 12:45:00.172751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.172658 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68d0e17-7808-425f-b9d3-ea2d4419db86" containerName="revoke-keys" Jul 21 12:45:00.172751 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.172669 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68d0e17-7808-425f-b9d3-ea2d4419db86" containerName="revoke-keys" Jul 21 12:45:00.172836 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.172765 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e68d0e17-7808-425f-b9d3-ea2d4419db86" containerName="revoke-keys" Jul 21 12:45:00.175914 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.175896 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" Jul 21 12:45:00.178209 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.178189 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2"] Jul 21 12:45:00.181772 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.181741 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" Jul 21 12:45:00.183284 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.183260 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:45:00.187196 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.187176 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" Jul 21 12:45:00.189147 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.189124 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl"] Jul 21 12:45:00.201176 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.201149 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" Jul 21 12:45:00.205686 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.205646 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2"] Jul 21 12:45:00.366938 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.366910 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl"] Jul 21 12:45:00.371098 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:45:00.371066 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b130b3f_835e_4ede_900b_e9f7aba78a71.slice/crio-91e322dfaedac43ff92d321a788a50624e551737df5a9cd624212599a15669fe WatchSource:0}: Error finding container 91e322dfaedac43ff92d321a788a50624e551737df5a9cd624212599a15669fe: Status 404 returned error can't find the container with id 91e322dfaedac43ff92d321a788a50624e551737df5a9cd624212599a15669fe Jul 21 12:45:00.399218 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.399195 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2"] Jul 21 12:45:00.401736 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:45:00.401709 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75cea34b_85a9_4115_b231_1b9afa788642.slice/crio-59cc1ef41285b59a3aad6b6056348cee46f4ea3718d0f1369a4ad386b80a8b8f WatchSource:0}: Error finding container 59cc1ef41285b59a3aad6b6056348cee46f4ea3718d0f1369a4ad386b80a8b8f: Status 404 returned error can't find the container with id 59cc1ef41285b59a3aad6b6056348cee46f4ea3718d0f1369a4ad386b80a8b8f Jul 21 12:45:00.826588 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.826463 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" event={"ID":"75cea34b-85a9-4115-b231-1b9afa788642","Type":"ContainerStarted","Data":"c44307e24f4a6a460cd78e40ff50b2cff989cd5b142f0892b7fae610be0d0e54"} Jul 21 12:45:00.826588 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.826526 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" event={"ID":"75cea34b-85a9-4115-b231-1b9afa788642","Type":"ContainerStarted","Data":"59cc1ef41285b59a3aad6b6056348cee46f4ea3718d0f1369a4ad386b80a8b8f"} Jul 21 12:45:00.828126 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.828102 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" event={"ID":"2b130b3f-835e-4ede-900b-e9f7aba78a71","Type":"ContainerStarted","Data":"185d802ecd36104d85e09d8f9718c01a61919b73fcdeac0254fef0427788c79c"} Jul 21 12:45:00.828267 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.828145 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" event={"ID":"2b130b3f-835e-4ede-900b-e9f7aba78a71","Type":"ContainerStarted","Data":"91e322dfaedac43ff92d321a788a50624e551737df5a9cd624212599a15669fe"} Jul 21 12:45:00.851932 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.851887 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" podStartSLOduration=0.851872006 podStartE2EDuration="851.872006ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:00.851857377 +0000 UTC m=+2034.345686370" watchObservedRunningTime="2026-07-21 12:45:00.851872006 +0000 UTC m=+2034.345700999" Jul 21 12:45:00.876160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:00.876112 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" podStartSLOduration=0.876098929 podStartE2EDuration="876.098929ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:00.875847332 +0000 UTC m=+2034.369676324" watchObservedRunningTime="2026-07-21 12:45:00.876098929 +0000 UTC m=+2034.369927924" Jul 21 12:45:01.835343 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:01.835311 2550 generic.go:356] "Generic (PLEG): container finished" podID="2b130b3f-835e-4ede-900b-e9f7aba78a71" containerID="185d802ecd36104d85e09d8f9718c01a61919b73fcdeac0254fef0427788c79c" exitCode=0 Jul 21 12:45:01.835794 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:01.835402 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" event={"ID":"2b130b3f-835e-4ede-900b-e9f7aba78a71","Type":"ContainerDied","Data":"185d802ecd36104d85e09d8f9718c01a61919b73fcdeac0254fef0427788c79c"} Jul 21 12:45:01.836800 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:01.836780 2550 generic.go:356] "Generic (PLEG): container finished" podID="75cea34b-85a9-4115-b231-1b9afa788642" containerID="c44307e24f4a6a460cd78e40ff50b2cff989cd5b142f0892b7fae610be0d0e54" exitCode=0 Jul 21 12:45:01.836910 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:01.836862 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" event={"ID":"75cea34b-85a9-4115-b231-1b9afa788642","Type":"ContainerDied","Data":"c44307e24f4a6a460cd78e40ff50b2cff989cd5b142f0892b7fae610be0d0e54"} Jul 21 12:45:02.258675 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:02.258645 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl"] Jul 21 12:45:02.990116 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:02.990086 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" Jul 21 12:45:03.013124 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.013101 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" Jul 21 12:45:03.849484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.849455 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" Jul 21 12:45:03.849484 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.849466 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl" event={"ID":"2b130b3f-835e-4ede-900b-e9f7aba78a71","Type":"ContainerDied","Data":"91e322dfaedac43ff92d321a788a50624e551737df5a9cd624212599a15669fe"} Jul 21 12:45:03.849746 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.849512 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91e322dfaedac43ff92d321a788a50624e551737df5a9cd624212599a15669fe" Jul 21 12:45:03.850852 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.850832 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" Jul 21 12:45:03.850958 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.850872 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743965-vcmq2" event={"ID":"75cea34b-85a9-4115-b231-1b9afa788642","Type":"ContainerDied","Data":"59cc1ef41285b59a3aad6b6056348cee46f4ea3718d0f1369a4ad386b80a8b8f"} Jul 21 12:45:03.850958 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.850911 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59cc1ef41285b59a3aad6b6056348cee46f4ea3718d0f1369a4ad386b80a8b8f" Jul 21 12:45:03.890000 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.889961 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl"] Jul 21 12:45:03.894036 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:03.894008 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-79283f7d-29743965-r95hl"] Jul 21 12:45:05.120531 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:05.120499 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b130b3f-835e-4ede-900b-e9f7aba78a71" path="/var/lib/kubelet/pods/2b130b3f-835e-4ede-900b-e9f7aba78a71/volumes" Jul 21 12:45:06.545798 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:06.545763 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq"] Jul 21 12:45:06.549320 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:06.549290 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-79283f7d-mzpwq"] Jul 21 12:45:07.121586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:07.121554 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e68d0e17-7808-425f-b9d3-ea2d4419db86" path="/var/lib/kubelet/pods/e68d0e17-7808-425f-b9d3-ea2d4419db86/volumes" Jul 21 12:45:35.828107 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828025 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm"] Jul 21 12:45:35.828577 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828503 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b130b3f-835e-4ede-900b-e9f7aba78a71" containerName="cleanup" Jul 21 12:45:35.828577 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828516 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b130b3f-835e-4ede-900b-e9f7aba78a71" containerName="cleanup" Jul 21 12:45:35.828577 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828530 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75cea34b-85a9-4115-b231-1b9afa788642" containerName="cleanup" Jul 21 12:45:35.828577 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828535 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="75cea34b-85a9-4115-b231-1b9afa788642" containerName="cleanup" Jul 21 12:45:35.828711 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828630 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="75cea34b-85a9-4115-b231-1b9afa788642" containerName="cleanup" Jul 21 12:45:35.828711 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.828640 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b130b3f-835e-4ede-900b-e9f7aba78a71" containerName="cleanup" Jul 21 12:45:35.831901 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.831885 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:35.835714 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.835694 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:45:35.841478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.841454 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm"] Jul 21 12:45:35.972871 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:35.972833 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm\" (UID: \"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:36.074101 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:36.074055 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm\" (UID: \"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:36.074734 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:36.074715 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm\" (UID: \"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:36.143689 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:36.143606 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:36.286309 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:36.286278 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm"] Jul 21 12:45:36.288123 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:45:36.288101 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b98628_eaf0_4a04_8aa3_ccb3a006d3a5.slice/crio-3975b3385e8d17d4b08a3e26bcaee69614630c06f5f4645cddbc41ae71eb60d3 WatchSource:0}: Error finding container 3975b3385e8d17d4b08a3e26bcaee69614630c06f5f4645cddbc41ae71eb60d3: Status 404 returned error can't find the container with id 3975b3385e8d17d4b08a3e26bcaee69614630c06f5f4645cddbc41ae71eb60d3 Jul 21 12:45:37.037109 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:37.037075 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" event={"ID":"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5","Type":"ContainerStarted","Data":"d1711ec3b8b0d08baecba63551562ec01d1cacb1350da02d03ec6e64bc45a299"} Jul 21 12:45:37.037531 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:37.037156 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" event={"ID":"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5","Type":"ContainerStarted","Data":"3975b3385e8d17d4b08a3e26bcaee69614630c06f5f4645cddbc41ae71eb60d3"} Jul 21 12:45:37.055364 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:37.055313 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" podStartSLOduration=2.055294431 podStartE2EDuration="2.055294431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:37.05477984 +0000 UTC m=+2070.548608833" watchObservedRunningTime="2026-07-21 12:45:37.055294431 +0000 UTC m=+2070.549123424" Jul 21 12:45:38.043628 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:38.043590 2550 generic.go:356] "Generic (PLEG): container finished" podID="e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5" containerID="d1711ec3b8b0d08baecba63551562ec01d1cacb1350da02d03ec6e64bc45a299" exitCode=0 Jul 21 12:45:38.044032 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:38.043677 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" event={"ID":"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5","Type":"ContainerDied","Data":"d1711ec3b8b0d08baecba63551562ec01d1cacb1350da02d03ec6e64bc45a299"} Jul 21 12:45:39.192834 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:39.192811 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:39.308095 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:39.308002 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5-maas-api-service-ca\") pod \"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5\" (UID: \"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5\") " Jul 21 12:45:39.308352 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:39.308327 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5" (UID: "e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:45:39.409545 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:39.409511 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:45:40.056099 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:40.056069 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" event={"ID":"e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5","Type":"ContainerDied","Data":"3975b3385e8d17d4b08a3e26bcaee69614630c06f5f4645cddbc41ae71eb60d3"} Jul 21 12:45:40.056099 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:40.056085 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm" Jul 21 12:45:40.056305 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:40.056114 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3975b3385e8d17d4b08a3e26bcaee69614630c06f5f4645cddbc41ae71eb60d3" Jul 21 12:45:50.908697 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:50.908663 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm"] Jul 21 12:45:50.911555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:50.911534 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-812e6bee-z8ppm"] Jul 21 12:45:51.121881 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:45:51.121853 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5" path="/var/lib/kubelet/pods/e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5/volumes" Jul 21 12:46:07.150612 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:46:07.150492 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:46:07.161711 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:46:07.161690 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:46:07.167031 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:46:07.167013 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:46:07.174836 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:46:07.174819 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:47:43.772198 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.772164 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx"] Jul 21 12:47:43.772838 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.772675 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5" containerName="revoke-keys" Jul 21 12:47:43.772838 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.772691 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5" containerName="revoke-keys" Jul 21 12:47:43.772838 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.772793 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5b98628-eaf0-4a04-8aa3-ccb3a006d3a5" containerName="revoke-keys" Jul 21 12:47:43.776034 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.776014 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:43.778501 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.778479 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:47:43.784016 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.783966 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx"] Jul 21 12:47:43.829630 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.829596 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d233b0-474f-415f-9395-9fd62d24cc5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx\" (UID: \"c6d233b0-474f-415f-9395-9fd62d24cc5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:43.930586 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.930539 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d233b0-474f-415f-9395-9fd62d24cc5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx\" (UID: \"c6d233b0-474f-415f-9395-9fd62d24cc5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:43.931227 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:43.931198 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d233b0-474f-415f-9395-9fd62d24cc5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx\" (UID: \"c6d233b0-474f-415f-9395-9fd62d24cc5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:44.088587 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:44.088498 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:44.436254 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:44.436221 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx"] Jul 21 12:47:44.439611 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:47:44.439585 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d233b0_474f_415f_9395_9fd62d24cc5e.slice/crio-522b63f96435e9557153f3dd84fce1efd3d9d15b0bfb131794e110480ce037b4 WatchSource:0}: Error finding container 522b63f96435e9557153f3dd84fce1efd3d9d15b0bfb131794e110480ce037b4: Status 404 returned error can't find the container with id 522b63f96435e9557153f3dd84fce1efd3d9d15b0bfb131794e110480ce037b4 Jul 21 12:47:44.783910 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:44.783878 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" event={"ID":"c6d233b0-474f-415f-9395-9fd62d24cc5e","Type":"ContainerStarted","Data":"522b63f96435e9557153f3dd84fce1efd3d9d15b0bfb131794e110480ce037b4"} Jul 21 12:47:45.791412 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:45.791374 2550 generic.go:356] "Generic (PLEG): container finished" podID="c6d233b0-474f-415f-9395-9fd62d24cc5e" containerID="5bdd01762f3a8388de650bb92f233973851eb9603e2918d33a7d7646ee98f5cc" exitCode=0 Jul 21 12:47:45.791880 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:45.791421 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" event={"ID":"c6d233b0-474f-415f-9395-9fd62d24cc5e","Type":"ContainerDied","Data":"5bdd01762f3a8388de650bb92f233973851eb9603e2918d33a7d7646ee98f5cc"} Jul 21 12:47:46.942192 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:46.942167 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:47.061063 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:47.060963 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d233b0-474f-415f-9395-9fd62d24cc5e-maas-api-service-ca\") pod \"c6d233b0-474f-415f-9395-9fd62d24cc5e\" (UID: \"c6d233b0-474f-415f-9395-9fd62d24cc5e\") " Jul 21 12:47:47.061332 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:47.061309 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d233b0-474f-415f-9395-9fd62d24cc5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6d233b0-474f-415f-9395-9fd62d24cc5e" (UID: "c6d233b0-474f-415f-9395-9fd62d24cc5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:47:47.162321 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:47.162289 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6d233b0-474f-415f-9395-9fd62d24cc5e-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:47:47.803765 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:47.803728 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" event={"ID":"c6d233b0-474f-415f-9395-9fd62d24cc5e","Type":"ContainerDied","Data":"522b63f96435e9557153f3dd84fce1efd3d9d15b0bfb131794e110480ce037b4"} Jul 21 12:47:47.803964 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:47.803786 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="522b63f96435e9557153f3dd84fce1efd3d9d15b0bfb131794e110480ce037b4" Jul 21 12:47:47.803964 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:47.803740 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx" Jul 21 12:47:58.848685 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:58.848649 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx"] Jul 21 12:47:58.851662 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:58.851639 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cd53929c-wp9kx"] Jul 21 12:47:59.121390 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:47:59.121303 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d233b0-474f-415f-9395-9fd62d24cc5e" path="/var/lib/kubelet/pods/c6d233b0-474f-415f-9395-9fd62d24cc5e/volumes" Jul 21 12:49:01.074668 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.074634 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm"] Jul 21 12:49:01.075165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.075075 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d233b0-474f-415f-9395-9fd62d24cc5e" containerName="revoke-keys" Jul 21 12:49:01.075165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.075085 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d233b0-474f-415f-9395-9fd62d24cc5e" containerName="revoke-keys" Jul 21 12:49:01.075165 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.075166 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d233b0-474f-415f-9395-9fd62d24cc5e" containerName="revoke-keys" Jul 21 12:49:01.078416 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.078400 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:01.081295 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.081274 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:49:01.086771 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.086707 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm"] Jul 21 12:49:01.198904 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.198851 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378eeee4-d433-433e-b27c-f11ec87be1e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm\" (UID: \"378eeee4-d433-433e-b27c-f11ec87be1e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:01.299914 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.299876 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378eeee4-d433-433e-b27c-f11ec87be1e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm\" (UID: \"378eeee4-d433-433e-b27c-f11ec87be1e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:01.300548 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.300529 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378eeee4-d433-433e-b27c-f11ec87be1e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm\" (UID: \"378eeee4-d433-433e-b27c-f11ec87be1e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:01.390294 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.390203 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:01.532782 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:01.532754 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm"] Jul 21 12:49:01.534993 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:49:01.534965 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378eeee4_d433_433e_b27c_f11ec87be1e8.slice/crio-8f3d1d6f89e58c6fd131f1156f3faf0e65b611c0f1c26bab542d5ecb27b48388 WatchSource:0}: Error finding container 8f3d1d6f89e58c6fd131f1156f3faf0e65b611c0f1c26bab542d5ecb27b48388: Status 404 returned error can't find the container with id 8f3d1d6f89e58c6fd131f1156f3faf0e65b611c0f1c26bab542d5ecb27b48388 Jul 21 12:49:02.251952 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:02.251922 2550 generic.go:356] "Generic (PLEG): container finished" podID="378eeee4-d433-433e-b27c-f11ec87be1e8" containerID="aea014c1110092bc13d4547d7261a802070a1ee378352196251e600946b90ade" exitCode=0 Jul 21 12:49:02.252312 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:02.252009 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" event={"ID":"378eeee4-d433-433e-b27c-f11ec87be1e8","Type":"ContainerDied","Data":"aea014c1110092bc13d4547d7261a802070a1ee378352196251e600946b90ade"} Jul 21 12:49:02.252312 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:02.252061 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" event={"ID":"378eeee4-d433-433e-b27c-f11ec87be1e8","Type":"ContainerStarted","Data":"8f3d1d6f89e58c6fd131f1156f3faf0e65b611c0f1c26bab542d5ecb27b48388"} Jul 21 12:49:03.402262 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:03.402239 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:03.521860 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:03.521773 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378eeee4-d433-433e-b27c-f11ec87be1e8-maas-api-service-ca\") pod \"378eeee4-d433-433e-b27c-f11ec87be1e8\" (UID: \"378eeee4-d433-433e-b27c-f11ec87be1e8\") " Jul 21 12:49:03.522160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:03.522133 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/378eeee4-d433-433e-b27c-f11ec87be1e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "378eeee4-d433-433e-b27c-f11ec87be1e8" (UID: "378eeee4-d433-433e-b27c-f11ec87be1e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:49:03.622869 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:03.622835 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378eeee4-d433-433e-b27c-f11ec87be1e8-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:49:04.266703 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:04.266671 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" Jul 21 12:49:04.266882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:04.266702 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm" event={"ID":"378eeee4-d433-433e-b27c-f11ec87be1e8","Type":"ContainerDied","Data":"8f3d1d6f89e58c6fd131f1156f3faf0e65b611c0f1c26bab542d5ecb27b48388"} Jul 21 12:49:04.266882 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:04.266746 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f3d1d6f89e58c6fd131f1156f3faf0e65b611c0f1c26bab542d5ecb27b48388" Jul 21 12:49:21.164453 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:21.164393 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm"] Jul 21 12:49:21.166620 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:21.166586 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffb6e94e-9kvrm"] Jul 21 12:49:23.120519 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:49:23.120489 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="378eeee4-d433-433e-b27c-f11ec87be1e8" path="/var/lib/kubelet/pods/378eeee4-d433-433e-b27c-f11ec87be1e8/volumes" Jul 21 12:50:07.254726 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:50:07.254691 2550 scope.go:117] "RemoveContainer" containerID="ab9c84fd86d71281e57937833ebeeda25acf9eb3873b5b7e1f1f7e83c56eee92" Jul 21 12:51:07.203209 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.203176 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:51:07.213254 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.213233 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:51:07.216503 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.216484 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:51:07.226107 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.226086 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:51:07.279701 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.279667 2550 scope.go:117] "RemoveContainer" containerID="185d802ecd36104d85e09d8f9718c01a61919b73fcdeac0254fef0427788c79c" Jul 21 12:51:07.290908 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.290886 2550 scope.go:117] "RemoveContainer" containerID="5a957f8e0d2c1636a76041c3117c66aaba2e797ccaf929152e6e52955deb72ce" Jul 21 12:51:07.298913 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:51:07.298895 2550 scope.go:117] "RemoveContainer" containerID="8b827740ecd7541bafd54b003c8f6343ba721c22a0d1a4b0bccb180f1516e2f4" Jul 21 12:52:07.327956 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:52:07.327922 2550 scope.go:117] "RemoveContainer" containerID="d1711ec3b8b0d08baecba63551562ec01d1cacb1350da02d03ec6e64bc45a299" Jul 21 12:53:03.910693 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.910656 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz"] Jul 21 12:53:03.911160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.911146 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="378eeee4-d433-433e-b27c-f11ec87be1e8" containerName="revoke-keys" Jul 21 12:53:03.911160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.911161 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="378eeee4-d433-433e-b27c-f11ec87be1e8" containerName="revoke-keys" Jul 21 12:53:03.911241 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.911226 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="378eeee4-d433-433e-b27c-f11ec87be1e8" containerName="revoke-keys" Jul 21 12:53:03.914438 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.914412 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:03.917187 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.917155 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:53:03.923005 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.922982 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz"] Jul 21 12:53:03.979604 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:03.979572 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0a25b78-4a8c-4b79-8762-39f2714e534a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz\" (UID: \"d0a25b78-4a8c-4b79-8762-39f2714e534a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:04.081176 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:04.081141 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0a25b78-4a8c-4b79-8762-39f2714e534a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz\" (UID: \"d0a25b78-4a8c-4b79-8762-39f2714e534a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:04.081822 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:04.081798 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0a25b78-4a8c-4b79-8762-39f2714e534a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz\" (UID: \"d0a25b78-4a8c-4b79-8762-39f2714e534a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:04.227243 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:04.227207 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:04.573382 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:04.573357 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz"] Jul 21 12:53:04.575235 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:53:04.575208 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a25b78_4a8c_4b79_8762_39f2714e534a.slice/crio-a80af1b4c27641c79e04045f94710488feecc9f25e49fc3b9755c222d06dbfac WatchSource:0}: Error finding container a80af1b4c27641c79e04045f94710488feecc9f25e49fc3b9755c222d06dbfac: Status 404 returned error can't find the container with id a80af1b4c27641c79e04045f94710488feecc9f25e49fc3b9755c222d06dbfac Jul 21 12:53:04.684144 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:04.684098 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" event={"ID":"d0a25b78-4a8c-4b79-8762-39f2714e534a","Type":"ContainerStarted","Data":"a80af1b4c27641c79e04045f94710488feecc9f25e49fc3b9755c222d06dbfac"} Jul 21 12:53:05.691060 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:05.691022 2550 generic.go:356] "Generic (PLEG): container finished" podID="d0a25b78-4a8c-4b79-8762-39f2714e534a" containerID="0ea9b7d0f8e9ac9405ba6f8767ca124667e64a948683161a1a551d42e1c3194e" exitCode=0 Jul 21 12:53:05.691450 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:05.691066 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" event={"ID":"d0a25b78-4a8c-4b79-8762-39f2714e534a","Type":"ContainerDied","Data":"0ea9b7d0f8e9ac9405ba6f8767ca124667e64a948683161a1a551d42e1c3194e"} Jul 21 12:53:06.842511 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:06.842484 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:06.910530 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:06.910501 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0a25b78-4a8c-4b79-8762-39f2714e534a-maas-api-service-ca\") pod \"d0a25b78-4a8c-4b79-8762-39f2714e534a\" (UID: \"d0a25b78-4a8c-4b79-8762-39f2714e534a\") " Jul 21 12:53:06.910868 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:06.910846 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0a25b78-4a8c-4b79-8762-39f2714e534a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0a25b78-4a8c-4b79-8762-39f2714e534a" (UID: "d0a25b78-4a8c-4b79-8762-39f2714e534a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:53:07.012272 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:07.012186 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0a25b78-4a8c-4b79-8762-39f2714e534a-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:53:07.703188 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:07.703161 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" Jul 21 12:53:07.703372 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:07.703204 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz" event={"ID":"d0a25b78-4a8c-4b79-8762-39f2714e534a","Type":"ContainerDied","Data":"a80af1b4c27641c79e04045f94710488feecc9f25e49fc3b9755c222d06dbfac"} Jul 21 12:53:07.703372 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:07.703250 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a80af1b4c27641c79e04045f94710488feecc9f25e49fc3b9755c222d06dbfac" Jul 21 12:53:19.014654 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:19.014589 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz"] Jul 21 12:53:19.017382 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:19.017361 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cb55b-dqzfz"] Jul 21 12:53:19.120758 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:19.120715 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0a25b78-4a8c-4b79-8762-39f2714e534a" path="/var/lib/kubelet/pods/d0a25b78-4a8c-4b79-8762-39f2714e534a/volumes" Jul 21 12:53:26.167024 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.166992 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z"] Jul 21 12:53:26.167575 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.167557 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0a25b78-4a8c-4b79-8762-39f2714e534a" containerName="revoke-keys" Jul 21 12:53:26.167626 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.167578 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0a25b78-4a8c-4b79-8762-39f2714e534a" containerName="revoke-keys" Jul 21 12:53:26.167706 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.167693 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0a25b78-4a8c-4b79-8762-39f2714e534a" containerName="revoke-keys" Jul 21 12:53:26.171267 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.171248 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:26.174454 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.174412 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:53:26.180995 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.180975 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z"] Jul 21 12:53:26.201313 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.201285 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z\" (UID: \"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:26.302193 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.302160 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z\" (UID: \"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:26.302873 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.302850 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z\" (UID: \"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:26.483051 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.483017 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:26.830981 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:26.830921 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z"] Jul 21 12:53:26.834548 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:53:26.834504 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afe19a4_fef5_4dbf_94f6_ad80eef7dad0.slice/crio-95dcb94667cafc14cb6df46d454d2af20e2dc14d76ca8db62c33ed7e9f7fb66b WatchSource:0}: Error finding container 95dcb94667cafc14cb6df46d454d2af20e2dc14d76ca8db62c33ed7e9f7fb66b: Status 404 returned error can't find the container with id 95dcb94667cafc14cb6df46d454d2af20e2dc14d76ca8db62c33ed7e9f7fb66b Jul 21 12:53:27.814713 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:27.814676 2550 generic.go:356] "Generic (PLEG): container finished" podID="1afe19a4-fef5-4dbf-94f6-ad80eef7dad0" containerID="0b74bd56c78d0e91a1a2ef66bce95750f8a0fd791dd18baeafc4f52ceec338d0" exitCode=0 Jul 21 12:53:27.815075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:27.814750 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" event={"ID":"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0","Type":"ContainerDied","Data":"0b74bd56c78d0e91a1a2ef66bce95750f8a0fd791dd18baeafc4f52ceec338d0"} Jul 21 12:53:27.815075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:27.814791 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" event={"ID":"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0","Type":"ContainerStarted","Data":"95dcb94667cafc14cb6df46d454d2af20e2dc14d76ca8db62c33ed7e9f7fb66b"} Jul 21 12:53:28.963614 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:28.963591 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:29.026229 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:29.026200 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0-maas-api-service-ca\") pod \"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0\" (UID: \"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0\") " Jul 21 12:53:29.026545 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:29.026524 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1afe19a4-fef5-4dbf-94f6-ad80eef7dad0" (UID: "1afe19a4-fef5-4dbf-94f6-ad80eef7dad0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:53:29.127143 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:29.127072 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:53:29.827979 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:29.827947 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" Jul 21 12:53:29.828170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:29.827980 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z" event={"ID":"1afe19a4-fef5-4dbf-94f6-ad80eef7dad0","Type":"ContainerDied","Data":"95dcb94667cafc14cb6df46d454d2af20e2dc14d76ca8db62c33ed7e9f7fb66b"} Jul 21 12:53:29.828170 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:29.828039 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95dcb94667cafc14cb6df46d454d2af20e2dc14d76ca8db62c33ed7e9f7fb66b" Jul 21 12:53:41.243896 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:41.243862 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z"] Jul 21 12:53:41.245936 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:41.245913 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ded38a-htg4z"] Jul 21 12:53:43.121006 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:53:43.120972 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1afe19a4-fef5-4dbf-94f6-ad80eef7dad0" path="/var/lib/kubelet/pods/1afe19a4-fef5-4dbf-94f6-ad80eef7dad0/volumes" Jul 21 12:54:07.358337 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:07.358304 2550 scope.go:117] "RemoveContainer" containerID="5bdd01762f3a8388de650bb92f233973851eb9603e2918d33a7d7646ee98f5cc" Jul 21 12:54:12.594736 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.594706 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m"] Jul 21 12:54:12.595202 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.595188 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afe19a4-fef5-4dbf-94f6-ad80eef7dad0" containerName="revoke-keys" Jul 21 12:54:12.595246 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.595206 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afe19a4-fef5-4dbf-94f6-ad80eef7dad0" containerName="revoke-keys" Jul 21 12:54:12.595289 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.595280 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1afe19a4-fef5-4dbf-94f6-ad80eef7dad0" containerName="revoke-keys" Jul 21 12:54:12.599796 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.599778 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:12.602851 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.602831 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:54:12.607187 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.607156 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m"] Jul 21 12:54:12.657103 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.657067 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5c86a1-c9b1-4956-901c-afdb2b27c84d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-635000-fj72m\" (UID: \"7d5c86a1-c9b1-4956-901c-afdb2b27c84d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:12.757878 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.757842 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5c86a1-c9b1-4956-901c-afdb2b27c84d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-635000-fj72m\" (UID: \"7d5c86a1-c9b1-4956-901c-afdb2b27c84d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:12.758512 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.758492 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5c86a1-c9b1-4956-901c-afdb2b27c84d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-635000-fj72m\" (UID: \"7d5c86a1-c9b1-4956-901c-afdb2b27c84d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:12.912669 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:12.912579 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:13.259130 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:13.259102 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m"] Jul 21 12:54:13.261410 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:54:13.261379 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5c86a1_c9b1_4956_901c_afdb2b27c84d.slice/crio-fce3ebf97d638319fe92278e1e7e372180d4685a1c53f12cf1ca758e30d5e7ca WatchSource:0}: Error finding container fce3ebf97d638319fe92278e1e7e372180d4685a1c53f12cf1ca758e30d5e7ca: Status 404 returned error can't find the container with id fce3ebf97d638319fe92278e1e7e372180d4685a1c53f12cf1ca758e30d5e7ca Jul 21 12:54:14.084547 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:14.084513 2550 generic.go:356] "Generic (PLEG): container finished" podID="7d5c86a1-c9b1-4956-901c-afdb2b27c84d" containerID="d5948a04350ce1783858b2ee76332ffbba41bca0b5c5fd551df2cdaa89611907" exitCode=0 Jul 21 12:54:14.084913 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:14.084603 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" event={"ID":"7d5c86a1-c9b1-4956-901c-afdb2b27c84d","Type":"ContainerDied","Data":"d5948a04350ce1783858b2ee76332ffbba41bca0b5c5fd551df2cdaa89611907"} Jul 21 12:54:14.084913 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:14.084656 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" event={"ID":"7d5c86a1-c9b1-4956-901c-afdb2b27c84d","Type":"ContainerStarted","Data":"fce3ebf97d638319fe92278e1e7e372180d4685a1c53f12cf1ca758e30d5e7ca"} Jul 21 12:54:15.233789 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:15.233761 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:15.282006 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:15.281969 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5c86a1-c9b1-4956-901c-afdb2b27c84d-maas-api-service-ca\") pod \"7d5c86a1-c9b1-4956-901c-afdb2b27c84d\" (UID: \"7d5c86a1-c9b1-4956-901c-afdb2b27c84d\") " Jul 21 12:54:15.282321 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:15.282299 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5c86a1-c9b1-4956-901c-afdb2b27c84d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d5c86a1-c9b1-4956-901c-afdb2b27c84d" (UID: "7d5c86a1-c9b1-4956-901c-afdb2b27c84d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:54:15.282374 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:15.282367 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5c86a1-c9b1-4956-901c-afdb2b27c84d-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:54:16.097993 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:16.097963 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" Jul 21 12:54:16.098160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:16.097986 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m" event={"ID":"7d5c86a1-c9b1-4956-901c-afdb2b27c84d","Type":"ContainerDied","Data":"fce3ebf97d638319fe92278e1e7e372180d4685a1c53f12cf1ca758e30d5e7ca"} Jul 21 12:54:16.098160 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:16.098078 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fce3ebf97d638319fe92278e1e7e372180d4685a1c53f12cf1ca758e30d5e7ca" Jul 21 12:54:32.684891 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:32.684805 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m"] Jul 21 12:54:32.687042 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:32.687019 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-635000-fj72m"] Jul 21 12:54:33.120530 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:33.120408 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d5c86a1-c9b1-4956-901c-afdb2b27c84d" path="/var/lib/kubelet/pods/7d5c86a1-c9b1-4956-901c-afdb2b27c84d/volumes" Jul 21 12:54:39.368455 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.368348 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt"] Jul 21 12:54:39.368835 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.368822 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5c86a1-c9b1-4956-901c-afdb2b27c84d" containerName="revoke-keys" Jul 21 12:54:39.368835 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.368836 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5c86a1-c9b1-4956-901c-afdb2b27c84d" containerName="revoke-keys" Jul 21 12:54:39.368915 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.368905 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5c86a1-c9b1-4956-901c-afdb2b27c84d" containerName="revoke-keys" Jul 21 12:54:39.373153 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.373136 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:39.383629 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.383610 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 12:54:39.393555 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.393528 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt"] Jul 21 12:54:39.513004 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.512971 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211eb960-5e7e-4999-8e4e-baf8cdb79ee0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8af997-wk4gt\" (UID: \"211eb960-5e7e-4999-8e4e-baf8cdb79ee0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:39.613988 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.613952 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211eb960-5e7e-4999-8e4e-baf8cdb79ee0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8af997-wk4gt\" (UID: \"211eb960-5e7e-4999-8e4e-baf8cdb79ee0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:39.614736 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.614698 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211eb960-5e7e-4999-8e4e-baf8cdb79ee0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8af997-wk4gt\" (UID: \"211eb960-5e7e-4999-8e4e-baf8cdb79ee0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:39.682710 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.682677 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:39.822009 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:39.821985 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt"] Jul 21 12:54:39.823953 ip-10-0-136-33 kubenswrapper[2550]: W0721 12:54:39.823918 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod211eb960_5e7e_4999_8e4e_baf8cdb79ee0.slice/crio-82ddf157b5f68daf9cedefbe61e817502ae7d3d46ccf4208f4f3f4b0af45d6f0 WatchSource:0}: Error finding container 82ddf157b5f68daf9cedefbe61e817502ae7d3d46ccf4208f4f3f4b0af45d6f0: Status 404 returned error can't find the container with id 82ddf157b5f68daf9cedefbe61e817502ae7d3d46ccf4208f4f3f4b0af45d6f0 Jul 21 12:54:40.235075 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:40.235042 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" event={"ID":"211eb960-5e7e-4999-8e4e-baf8cdb79ee0","Type":"ContainerStarted","Data":"6a41286276ecb9cecbc3a13e538a85a5ac72908ce9c186ee81f5c13dc24143a2"} Jul 21 12:54:40.235273 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:40.235089 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" event={"ID":"211eb960-5e7e-4999-8e4e-baf8cdb79ee0","Type":"ContainerStarted","Data":"82ddf157b5f68daf9cedefbe61e817502ae7d3d46ccf4208f4f3f4b0af45d6f0"} Jul 21 12:54:40.260519 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:40.260464 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" podStartSLOduration=1.260412111 podStartE2EDuration="1.260412111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:54:40.258821484 +0000 UTC m=+2613.752650477" watchObservedRunningTime="2026-07-21 12:54:40.260412111 +0000 UTC m=+2613.754241104" Jul 21 12:54:41.248955 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:41.248908 2550 generic.go:356] "Generic (PLEG): container finished" podID="211eb960-5e7e-4999-8e4e-baf8cdb79ee0" containerID="6a41286276ecb9cecbc3a13e538a85a5ac72908ce9c186ee81f5c13dc24143a2" exitCode=0 Jul 21 12:54:41.249349 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:41.248984 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" event={"ID":"211eb960-5e7e-4999-8e4e-baf8cdb79ee0","Type":"ContainerDied","Data":"6a41286276ecb9cecbc3a13e538a85a5ac72908ce9c186ee81f5c13dc24143a2"} Jul 21 12:54:42.398864 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:42.398838 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:42.545413 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:42.545328 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211eb960-5e7e-4999-8e4e-baf8cdb79ee0-maas-api-service-ca\") pod \"211eb960-5e7e-4999-8e4e-baf8cdb79ee0\" (UID: \"211eb960-5e7e-4999-8e4e-baf8cdb79ee0\") " Jul 21 12:54:42.545700 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:42.545677 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/211eb960-5e7e-4999-8e4e-baf8cdb79ee0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "211eb960-5e7e-4999-8e4e-baf8cdb79ee0" (UID: "211eb960-5e7e-4999-8e4e-baf8cdb79ee0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:54:42.646356 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:42.646314 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211eb960-5e7e-4999-8e4e-baf8cdb79ee0-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 12:54:43.262460 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:43.262414 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" event={"ID":"211eb960-5e7e-4999-8e4e-baf8cdb79ee0","Type":"ContainerDied","Data":"82ddf157b5f68daf9cedefbe61e817502ae7d3d46ccf4208f4f3f4b0af45d6f0"} Jul 21 12:54:43.262639 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:43.262472 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt" Jul 21 12:54:43.262639 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:43.262479 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82ddf157b5f68daf9cedefbe61e817502ae7d3d46ccf4208f4f3f4b0af45d6f0" Jul 21 12:54:59.455371 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:59.455334 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt"] Jul 21 12:54:59.457694 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:54:59.457668 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8af997-wk4gt"] Jul 21 12:55:01.120285 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:55:01.120246 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="211eb960-5e7e-4999-8e4e-baf8cdb79ee0" path="/var/lib/kubelet/pods/211eb960-5e7e-4999-8e4e-baf8cdb79ee0/volumes" Jul 21 12:55:07.381392 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:55:07.381357 2550 scope.go:117] "RemoveContainer" containerID="aea014c1110092bc13d4547d7261a802070a1ee378352196251e600946b90ade" Jul 21 12:56:07.240649 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:56:07.240545 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:56:07.249203 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:56:07.249184 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:56:07.255359 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:56:07.255341 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 12:56:07.265101 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:56:07.265082 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 12:59:07.429478 ip-10-0-136-33 kubenswrapper[2550]: I0721 12:59:07.429446 2550 scope.go:117] "RemoveContainer" containerID="0ea9b7d0f8e9ac9405ba6f8767ca124667e64a948683161a1a551d42e1c3194e" Jul 21 13:00:00.137136 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.137100 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d"] Jul 21 13:00:00.137699 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.137669 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="211eb960-5e7e-4999-8e4e-baf8cdb79ee0" containerName="revoke-keys" Jul 21 13:00:00.137699 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.137686 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="211eb960-5e7e-4999-8e4e-baf8cdb79ee0" containerName="revoke-keys" Jul 21 13:00:00.137845 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.137784 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="211eb960-5e7e-4999-8e4e-baf8cdb79ee0" containerName="revoke-keys" Jul 21 13:00:00.140943 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.140926 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b"] Jul 21 13:00:00.141058 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.141038 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" Jul 21 13:00:00.144172 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.144153 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" Jul 21 13:00:00.144724 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.144705 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 13:00:00.147929 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.147910 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2"] Jul 21 13:00:00.156296 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.156268 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" Jul 21 13:00:00.163762 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.163730 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d"] Jul 21 13:00:00.163870 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.163786 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b"] Jul 21 13:00:00.163929 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.163903 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" Jul 21 13:00:00.165758 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.165733 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2"] Jul 21 13:00:00.171047 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.171024 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" Jul 21 13:00:00.205953 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.205927 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" Jul 21 13:00:00.367318 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.367288 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b"] Jul 21 13:00:00.369602 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:00:00.369577 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b83db6_3e34_4601_9e85_5309136eba00.slice/crio-af40453b6488b00343d40199bbe747d6693a900941e8b71e0f722bff266471f5 WatchSource:0}: Error finding container af40453b6488b00343d40199bbe747d6693a900941e8b71e0f722bff266471f5: Status 404 returned error can't find the container with id af40453b6488b00343d40199bbe747d6693a900941e8b71e0f722bff266471f5 Jul 21 13:00:00.395810 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.395782 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2"] Jul 21 13:00:00.395958 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:00:00.395930 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded77dcaf_e0c2_46f7_8ece_da47934963b3.slice/crio-0c703cc06968bc6685cb0938ed9a271ea91ba5e5da721091693a4423101354dd WatchSource:0}: Error finding container 0c703cc06968bc6685cb0938ed9a271ea91ba5e5da721091693a4423101354dd: Status 404 returned error can't find the container with id 0c703cc06968bc6685cb0938ed9a271ea91ba5e5da721091693a4423101354dd Jul 21 13:00:00.571211 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:00.571184 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d"] Jul 21 13:00:00.573341 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:00:00.573314 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21af0ca4_55f4_4c6e_af07_928d9962929a.slice/crio-dc57f6631d98f878519697650060dcb7eef0aa280524a197beb81c2685f467bb WatchSource:0}: Error finding container dc57f6631d98f878519697650060dcb7eef0aa280524a197beb81c2685f467bb: Status 404 returned error can't find the container with id dc57f6631d98f878519697650060dcb7eef0aa280524a197beb81c2685f467bb Jul 21 13:00:01.080128 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.080026 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" event={"ID":"ed77dcaf-e0c2-46f7-8ece-da47934963b3","Type":"ContainerStarted","Data":"b4fe003c76686874be938eaee111e7cb7f179728593f3bdaa0e4823b8ffd5f45"} Jul 21 13:00:01.080128 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.080096 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" event={"ID":"ed77dcaf-e0c2-46f7-8ece-da47934963b3","Type":"ContainerStarted","Data":"0c703cc06968bc6685cb0938ed9a271ea91ba5e5da721091693a4423101354dd"} Jul 21 13:00:01.081736 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.081712 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" event={"ID":"55b83db6-3e34-4601-9e85-5309136eba00","Type":"ContainerStarted","Data":"ae2500ed94718c0eead77d83c5ac9e94c8993673360442b180e28f0c9821493f"} Jul 21 13:00:01.081854 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.081765 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" event={"ID":"55b83db6-3e34-4601-9e85-5309136eba00","Type":"ContainerStarted","Data":"af40453b6488b00343d40199bbe747d6693a900941e8b71e0f722bff266471f5"} Jul 21 13:00:01.083292 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.083271 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" event={"ID":"21af0ca4-55f4-4c6e-af07-928d9962929a","Type":"ContainerStarted","Data":"71e14b12338f4d2bd5d37f2616e3da4ce154145d4d27819ffcd199ec910c9d21"} Jul 21 13:00:01.083413 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.083319 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" event={"ID":"21af0ca4-55f4-4c6e-af07-928d9962929a","Type":"ContainerStarted","Data":"dc57f6631d98f878519697650060dcb7eef0aa280524a197beb81c2685f467bb"} Jul 21 13:00:01.098820 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.098761 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" podStartSLOduration=1.098747864 podStartE2EDuration="1.098747864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:01.097280012 +0000 UTC m=+2934.591109007" watchObservedRunningTime="2026-07-21 13:00:01.098747864 +0000 UTC m=+2934.592576856" Jul 21 13:00:01.116169 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.116109 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" podStartSLOduration=1.116090657 podStartE2EDuration="1.116090657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:01.112499204 +0000 UTC m=+2934.606328197" watchObservedRunningTime="2026-07-21 13:00:01.116090657 +0000 UTC m=+2934.609919650" Jul 21 13:00:01.128384 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:01.128327 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" podStartSLOduration=1.12830916 podStartE2EDuration="1.12830916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:01.127403875 +0000 UTC m=+2934.621232870" watchObservedRunningTime="2026-07-21 13:00:01.12830916 +0000 UTC m=+2934.622138153" Jul 21 13:00:02.090017 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:02.089979 2550 generic.go:356] "Generic (PLEG): container finished" podID="ed77dcaf-e0c2-46f7-8ece-da47934963b3" containerID="b4fe003c76686874be938eaee111e7cb7f179728593f3bdaa0e4823b8ffd5f45" exitCode=0 Jul 21 13:00:02.090501 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:02.090066 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" event={"ID":"ed77dcaf-e0c2-46f7-8ece-da47934963b3","Type":"ContainerDied","Data":"b4fe003c76686874be938eaee111e7cb7f179728593f3bdaa0e4823b8ffd5f45"} Jul 21 13:00:02.091575 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:02.091550 2550 generic.go:356] "Generic (PLEG): container finished" podID="55b83db6-3e34-4601-9e85-5309136eba00" containerID="ae2500ed94718c0eead77d83c5ac9e94c8993673360442b180e28f0c9821493f" exitCode=0 Jul 21 13:00:02.091667 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:02.091637 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" event={"ID":"55b83db6-3e34-4601-9e85-5309136eba00","Type":"ContainerDied","Data":"ae2500ed94718c0eead77d83c5ac9e94c8993673360442b180e28f0c9821493f"} Jul 21 13:00:02.093000 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:02.092982 2550 generic.go:356] "Generic (PLEG): container finished" podID="21af0ca4-55f4-4c6e-af07-928d9962929a" containerID="71e14b12338f4d2bd5d37f2616e3da4ce154145d4d27819ffcd199ec910c9d21" exitCode=0 Jul 21 13:00:02.093080 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:02.093027 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" event={"ID":"21af0ca4-55f4-4c6e-af07-928d9962929a","Type":"ContainerDied","Data":"71e14b12338f4d2bd5d37f2616e3da4ce154145d4d27819ffcd199ec910c9d21"} Jul 21 13:00:03.267336 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:03.267310 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" Jul 21 13:00:03.295576 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:03.295553 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" Jul 21 13:00:03.321645 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:03.321620 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" Jul 21 13:00:04.106087 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.106057 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" Jul 21 13:00:04.106267 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.106095 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d" event={"ID":"21af0ca4-55f4-4c6e-af07-928d9962929a","Type":"ContainerDied","Data":"dc57f6631d98f878519697650060dcb7eef0aa280524a197beb81c2685f467bb"} Jul 21 13:00:04.106267 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.106143 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc57f6631d98f878519697650060dcb7eef0aa280524a197beb81c2685f467bb" Jul 21 13:00:04.107444 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.107413 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" Jul 21 13:00:04.107590 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.107455 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2" event={"ID":"ed77dcaf-e0c2-46f7-8ece-da47934963b3","Type":"ContainerDied","Data":"0c703cc06968bc6685cb0938ed9a271ea91ba5e5da721091693a4423101354dd"} Jul 21 13:00:04.107590 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.107494 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c703cc06968bc6685cb0938ed9a271ea91ba5e5da721091693a4423101354dd" Jul 21 13:00:04.108767 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.108743 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" event={"ID":"55b83db6-3e34-4601-9e85-5309136eba00","Type":"ContainerDied","Data":"af40453b6488b00343d40199bbe747d6693a900941e8b71e0f722bff266471f5"} Jul 21 13:00:04.108767 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.108769 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-pg58b" Jul 21 13:00:04.108942 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:04.108779 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af40453b6488b00343d40199bbe747d6693a900941e8b71e0f722bff266471f5" Jul 21 13:00:07.451978 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:00:07.451945 2550 scope.go:117] "RemoveContainer" containerID="0b74bd56c78d0e91a1a2ef66bce95750f8a0fd791dd18baeafc4f52ceec338d0" Jul 21 13:01:04.749443 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749389 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7349-140520293850032"] Jul 21 13:01:04.749948 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749860 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21af0ca4-55f4-4c6e-af07-928d9962929a" containerName="cleanup" Jul 21 13:01:04.749948 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749873 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="21af0ca4-55f4-4c6e-af07-928d9962929a" containerName="cleanup" Jul 21 13:01:04.749948 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749889 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed77dcaf-e0c2-46f7-8ece-da47934963b3" containerName="cleanup" Jul 21 13:01:04.749948 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749895 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed77dcaf-e0c2-46f7-8ece-da47934963b3" containerName="cleanup" Jul 21 13:01:04.749948 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749904 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b83db6-3e34-4601-9e85-5309136eba00" containerName="cleanup" Jul 21 13:01:04.749948 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749910 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b83db6-3e34-4601-9e85-5309136eba00" containerName="cleanup" Jul 21 13:01:04.750166 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749981 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b83db6-3e34-4601-9e85-5309136eba00" containerName="cleanup" Jul 21 13:01:04.750166 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749991 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="21af0ca4-55f4-4c6e-af07-928d9962929a" containerName="cleanup" Jul 21 13:01:04.750166 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.749999 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed77dcaf-e0c2-46f7-8ece-da47934963b3" containerName="cleanup" Jul 21 13:01:04.753290 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.753273 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:04.755962 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.755945 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h4fmk\"" Jul 21 13:01:04.760043 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.760019 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293850032"] Jul 21 13:01:04.905165 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:04.905121 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8d28\" (UniqueName: \"kubernetes.io/projected/b7445da5-2b08-49f3-af14-f619471f941c-kube-api-access-m8d28\") pod \"test-curl-7349-140520293850032\" (UID: \"b7445da5-2b08-49f3-af14-f619471f941c\") " pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:05.006163 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:05.006071 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8d28\" (UniqueName: \"kubernetes.io/projected/b7445da5-2b08-49f3-af14-f619471f941c-kube-api-access-m8d28\") pod \"test-curl-7349-140520293850032\" (UID: \"b7445da5-2b08-49f3-af14-f619471f941c\") " pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:05.015049 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:05.015016 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8d28\" (UniqueName: \"kubernetes.io/projected/b7445da5-2b08-49f3-af14-f619471f941c-kube-api-access-m8d28\") pod \"test-curl-7349-140520293850032\" (UID: \"b7445da5-2b08-49f3-af14-f619471f941c\") " pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:05.064865 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:05.064830 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:05.211180 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:05.211152 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293850032"] Jul 21 13:01:05.212996 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:05.212971 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7445da5_2b08_49f3_af14_f619471f941c.slice/crio-3e8b70fa75a96539608f33fe4cfece6cb59c5c7e42892f57edf419aadc7e23a5 WatchSource:0}: Error finding container 3e8b70fa75a96539608f33fe4cfece6cb59c5c7e42892f57edf419aadc7e23a5: Status 404 returned error can't find the container with id 3e8b70fa75a96539608f33fe4cfece6cb59c5c7e42892f57edf419aadc7e23a5 Jul 21 13:01:05.214640 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:05.214623 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:01:05.458408 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:05.458365 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293850032" event={"ID":"b7445da5-2b08-49f3-af14-f619471f941c","Type":"ContainerStarted","Data":"3e8b70fa75a96539608f33fe4cfece6cb59c5c7e42892f57edf419aadc7e23a5"} Jul 21 13:01:06.465469 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:06.465406 2550 generic.go:356] "Generic (PLEG): container finished" podID="b7445da5-2b08-49f3-af14-f619471f941c" containerID="7cba02bebe7b82d23fc7d00e67ebc8645696fcb9ee252e41da13e9e7cd207b3f" exitCode=0 Jul 21 13:01:06.465862 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:06.465462 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293850032" event={"ID":"b7445da5-2b08-49f3-af14-f619471f941c","Type":"ContainerDied","Data":"7cba02bebe7b82d23fc7d00e67ebc8645696fcb9ee252e41da13e9e7cd207b3f"} Jul 21 13:01:07.280373 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.280259 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 13:01:07.295586 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.289119 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 13:01:07.296107 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.296091 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 13:01:07.303576 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.303557 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xwhch_1c0acb4d-b957-4a5a-bc36-a077512a9cb3/ovn-acl-logging/0.log" Jul 21 13:01:07.473349 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.473319 2550 scope.go:117] "RemoveContainer" containerID="6a41286276ecb9cecbc3a13e538a85a5ac72908ce9c186ee81f5c13dc24143a2" Jul 21 13:01:07.482334 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.482306 2550 scope.go:117] "RemoveContainer" containerID="d5948a04350ce1783858b2ee76332ffbba41bca0b5c5fd551df2cdaa89611907" Jul 21 13:01:07.615703 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.615680 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:07.655722 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.655691 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7349-140520293850032_b7445da5-2b08-49f3-af14-f619471f941c/test-curl-7349-140520293850032/0.log" Jul 21 13:01:07.680904 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.680875 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7349-140520293850032"] Jul 21 13:01:07.684617 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.684595 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7349-140520293850032"] Jul 21 13:01:07.732778 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.732751 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8d28\" (UniqueName: \"kubernetes.io/projected/b7445da5-2b08-49f3-af14-f619471f941c-kube-api-access-m8d28\") pod \"b7445da5-2b08-49f3-af14-f619471f941c\" (UID: \"b7445da5-2b08-49f3-af14-f619471f941c\") " Jul 21 13:01:07.734809 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.734784 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7445da5-2b08-49f3-af14-f619471f941c-kube-api-access-m8d28" (OuterVolumeSpecName: "kube-api-access-m8d28") pod "b7445da5-2b08-49f3-af14-f619471f941c" (UID: "b7445da5-2b08-49f3-af14-f619471f941c"). InnerVolumeSpecName "kube-api-access-m8d28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:01:07.801514 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.801481 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7349-140520293850160"] Jul 21 13:01:07.802005 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.801985 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7445da5-2b08-49f3-af14-f619471f941c" containerName="test-curl-7349-140520293850032" Jul 21 13:01:07.802100 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.802007 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7445da5-2b08-49f3-af14-f619471f941c" containerName="test-curl-7349-140520293850032" Jul 21 13:01:07.802154 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.802115 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7445da5-2b08-49f3-af14-f619471f941c" containerName="test-curl-7349-140520293850032" Jul 21 13:01:07.805275 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.805255 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:07.810156 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.810130 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293850160"] Jul 21 13:01:07.834140 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.834106 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8d28\" (UniqueName: \"kubernetes.io/projected/b7445da5-2b08-49f3-af14-f619471f941c-kube-api-access-m8d28\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:07.934630 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:07.934595 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpk2\" (UniqueName: \"kubernetes.io/projected/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb-kube-api-access-5kpk2\") pod \"test-curl-7349-140520293850160\" (UID: \"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb\") " pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:08.036074 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.036037 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpk2\" (UniqueName: \"kubernetes.io/projected/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb-kube-api-access-5kpk2\") pod \"test-curl-7349-140520293850160\" (UID: \"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb\") " pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:08.045740 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.045716 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpk2\" (UniqueName: \"kubernetes.io/projected/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb-kube-api-access-5kpk2\") pod \"test-curl-7349-140520293850160\" (UID: \"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb\") " pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:08.115905 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.115866 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:08.254483 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.254456 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293850160"] Jul 21 13:01:08.255970 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:08.255939 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2b67f60_aa96_4cf7_9b93_e9d0ab6a55fb.slice/crio-373dc6b1ddc4b35825f7fde0cdcd4bb8cd413e4d7db218459e4e1886ed559f6c WatchSource:0}: Error finding container 373dc6b1ddc4b35825f7fde0cdcd4bb8cd413e4d7db218459e4e1886ed559f6c: Status 404 returned error can't find the container with id 373dc6b1ddc4b35825f7fde0cdcd4bb8cd413e4d7db218459e4e1886ed559f6c Jul 21 13:01:08.480140 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.480056 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293850160" event={"ID":"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb","Type":"ContainerStarted","Data":"373dc6b1ddc4b35825f7fde0cdcd4bb8cd413e4d7db218459e4e1886ed559f6c"} Jul 21 13:01:08.481270 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.481255 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:08.481270 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:08.481265 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e8b70fa75a96539608f33fe4cfece6cb59c5c7e42892f57edf419aadc7e23a5" Jul 21 13:01:08.491751 ip-10-0-136-33 kubenswrapper[2550]: E0721 13:01:08.491724 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7349-140520293850032\" is forbidden: User \"system:node:ip-10-0-136-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-33.ec2.internal' and this object" podUID="b7445da5-2b08-49f3-af14-f619471f941c" pod="opendatahub/test-curl-7349-140520293850032" Jul 21 13:01:09.122664 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:09.122629 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7445da5-2b08-49f3-af14-f619471f941c" path="/var/lib/kubelet/pods/b7445da5-2b08-49f3-af14-f619471f941c/volumes" Jul 21 13:01:09.490030 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:09.489995 2550 generic.go:356] "Generic (PLEG): container finished" podID="e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb" containerID="1e5056c84e3b51ee5a91eb41b93f47f7c8117818a855875f7ab7e479e961e7b1" exitCode=0 Jul 21 13:01:09.490410 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:09.490056 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293850160" event={"ID":"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb","Type":"ContainerDied","Data":"1e5056c84e3b51ee5a91eb41b93f47f7c8117818a855875f7ab7e479e961e7b1"} Jul 21 13:01:10.639529 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.639506 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:10.658908 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.658879 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7349-140520293850160_e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb/test-curl-7349-140520293850160/0.log" Jul 21 13:01:10.682300 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.682263 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7349-140520293850160"] Jul 21 13:01:10.685312 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.685283 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7349-140520293850160"] Jul 21 13:01:10.763275 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.763194 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kpk2\" (UniqueName: \"kubernetes.io/projected/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb-kube-api-access-5kpk2\") pod \"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb\" (UID: \"e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb\") " Jul 21 13:01:10.765173 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.765148 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb-kube-api-access-5kpk2" (OuterVolumeSpecName: "kube-api-access-5kpk2") pod "e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb" (UID: "e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb"). InnerVolumeSpecName "kube-api-access-5kpk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:01:10.832247 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.832213 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:10.832686 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.832671 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb" containerName="test-curl-7349-140520293850160" Jul 21 13:01:10.832750 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.832687 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb" containerName="test-curl-7349-140520293850160" Jul 21 13:01:10.832790 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.832765 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb" containerName="test-curl-7349-140520293850160" Jul 21 13:01:10.835861 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.835845 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:10.840880 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.840855 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:10.864305 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.864274 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kpk2\" (UniqueName: \"kubernetes.io/projected/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb-kube-api-access-5kpk2\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:10.965087 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:10.965055 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfwkw\" (UniqueName: \"kubernetes.io/projected/649045fe-3f4b-4831-9bf8-fbb956841856-kube-api-access-pfwkw\") pod \"test-curl-7349-140520293987360\" (UID: \"649045fe-3f4b-4831-9bf8-fbb956841856\") " pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:11.066151 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.066057 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfwkw\" (UniqueName: \"kubernetes.io/projected/649045fe-3f4b-4831-9bf8-fbb956841856-kube-api-access-pfwkw\") pod \"test-curl-7349-140520293987360\" (UID: \"649045fe-3f4b-4831-9bf8-fbb956841856\") " pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:11.074760 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.074729 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfwkw\" (UniqueName: \"kubernetes.io/projected/649045fe-3f4b-4831-9bf8-fbb956841856-kube-api-access-pfwkw\") pod \"test-curl-7349-140520293987360\" (UID: \"649045fe-3f4b-4831-9bf8-fbb956841856\") " pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:11.121170 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.121137 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb" path="/var/lib/kubelet/pods/e2b67f60-aa96-4cf7-9b93-e9d0ab6a55fb/volumes" Jul 21 13:01:11.146583 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.146554 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:11.286804 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.286659 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:11.289527 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:11.289500 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod649045fe_3f4b_4831_9bf8_fbb956841856.slice/crio-5cb24fd8820f4ee538d1671d86bb839834256676dd5fb00480340f6a5a127737 WatchSource:0}: Error finding container 5cb24fd8820f4ee538d1671d86bb839834256676dd5fb00480340f6a5a127737: Status 404 returned error can't find the container with id 5cb24fd8820f4ee538d1671d86bb839834256676dd5fb00480340f6a5a127737 Jul 21 13:01:11.503719 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.503690 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293850160" Jul 21 13:01:11.503921 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.503755 2550 scope.go:117] "RemoveContainer" containerID="1e5056c84e3b51ee5a91eb41b93f47f7c8117818a855875f7ab7e479e961e7b1" Jul 21 13:01:11.505215 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:11.505088 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293987360" event={"ID":"649045fe-3f4b-4831-9bf8-fbb956841856","Type":"ContainerStarted","Data":"5cb24fd8820f4ee538d1671d86bb839834256676dd5fb00480340f6a5a127737"} Jul 21 13:01:12.513340 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:12.513304 2550 generic.go:356] "Generic (PLEG): container finished" podID="649045fe-3f4b-4831-9bf8-fbb956841856" containerID="a26ef856eb80ffe3d1d49d8f6df672a103c54015a235559327f8c170718b96eb" exitCode=0 Jul 21 13:01:12.513809 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:12.513362 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293987360" event={"ID":"649045fe-3f4b-4831-9bf8-fbb956841856","Type":"ContainerDied","Data":"a26ef856eb80ffe3d1d49d8f6df672a103c54015a235559327f8c170718b96eb"} Jul 21 13:01:13.662999 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.662974 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:13.683286 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.683257 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7349-140520293987360_649045fe-3f4b-4831-9bf8-fbb956841856/test-curl-7349-140520293987360/0.log" Jul 21 13:01:13.705806 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.705770 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:13.709455 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.709415 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:13.794455 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.794346 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfwkw\" (UniqueName: \"kubernetes.io/projected/649045fe-3f4b-4831-9bf8-fbb956841856-kube-api-access-pfwkw\") pod \"649045fe-3f4b-4831-9bf8-fbb956841856\" (UID: \"649045fe-3f4b-4831-9bf8-fbb956841856\") " Jul 21 13:01:13.796408 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.796388 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/649045fe-3f4b-4831-9bf8-fbb956841856-kube-api-access-pfwkw" (OuterVolumeSpecName: "kube-api-access-pfwkw") pod "649045fe-3f4b-4831-9bf8-fbb956841856" (UID: "649045fe-3f4b-4831-9bf8-fbb956841856"). InnerVolumeSpecName "kube-api-access-pfwkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:01:13.826240 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.826212 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:13.826756 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.826740 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="649045fe-3f4b-4831-9bf8-fbb956841856" containerName="test-curl-7349-140520293987360" Jul 21 13:01:13.826810 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.826758 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="649045fe-3f4b-4831-9bf8-fbb956841856" containerName="test-curl-7349-140520293987360" Jul 21 13:01:13.826848 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.826838 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="649045fe-3f4b-4831-9bf8-fbb956841856" containerName="test-curl-7349-140520293987360" Jul 21 13:01:13.830196 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.830175 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:13.832736 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.832710 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:13.896097 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.896062 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfwkw\" (UniqueName: \"kubernetes.io/projected/649045fe-3f4b-4831-9bf8-fbb956841856-kube-api-access-pfwkw\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:13.997411 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:13.997374 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftrr7\" (UniqueName: \"kubernetes.io/projected/8b20d9da-d0e3-423e-b3d6-c2220410b0d0-kube-api-access-ftrr7\") pod \"test-curl-7349-140520293987360\" (UID: \"8b20d9da-d0e3-423e-b3d6-c2220410b0d0\") " pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:14.098604 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.098516 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftrr7\" (UniqueName: \"kubernetes.io/projected/8b20d9da-d0e3-423e-b3d6-c2220410b0d0-kube-api-access-ftrr7\") pod \"test-curl-7349-140520293987360\" (UID: \"8b20d9da-d0e3-423e-b3d6-c2220410b0d0\") " pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:14.106883 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.106859 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftrr7\" (UniqueName: \"kubernetes.io/projected/8b20d9da-d0e3-423e-b3d6-c2220410b0d0-kube-api-access-ftrr7\") pod \"test-curl-7349-140520293987360\" (UID: \"8b20d9da-d0e3-423e-b3d6-c2220410b0d0\") " pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:14.141035 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.140999 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:14.276538 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.276514 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:14.278035 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:14.278008 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b20d9da_d0e3_423e_b3d6_c2220410b0d0.slice/crio-6a7a36e7207b8ae7dc9a95e7908c7b8e66fd549c8af45ed902c58ac347940e29 WatchSource:0}: Error finding container 6a7a36e7207b8ae7dc9a95e7908c7b8e66fd549c8af45ed902c58ac347940e29: Status 404 returned error can't find the container with id 6a7a36e7207b8ae7dc9a95e7908c7b8e66fd549c8af45ed902c58ac347940e29 Jul 21 13:01:14.525206 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.525177 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:14.525401 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.525178 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5cb24fd8820f4ee538d1671d86bb839834256676dd5fb00480340f6a5a127737" Jul 21 13:01:14.526419 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.526390 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293987360" event={"ID":"8b20d9da-d0e3-423e-b3d6-c2220410b0d0","Type":"ContainerStarted","Data":"6a7a36e7207b8ae7dc9a95e7908c7b8e66fd549c8af45ed902c58ac347940e29"} Jul 21 13:01:14.537061 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:14.537032 2550 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7349-140520293987360" oldPodUID="649045fe-3f4b-4831-9bf8-fbb956841856" podUID="8b20d9da-d0e3-423e-b3d6-c2220410b0d0" Jul 21 13:01:15.121461 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:15.121412 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="649045fe-3f4b-4831-9bf8-fbb956841856" path="/var/lib/kubelet/pods/649045fe-3f4b-4831-9bf8-fbb956841856/volumes" Jul 21 13:01:15.533469 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:15.533411 2550 generic.go:356] "Generic (PLEG): container finished" podID="8b20d9da-d0e3-423e-b3d6-c2220410b0d0" containerID="fa4b4f7a61c0cdffbe760794055fde8c7576a860a9251cf0e17070d7f23a2936" exitCode=0 Jul 21 13:01:15.533655 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:15.533500 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293987360" event={"ID":"8b20d9da-d0e3-423e-b3d6-c2220410b0d0","Type":"ContainerDied","Data":"fa4b4f7a61c0cdffbe760794055fde8c7576a860a9251cf0e17070d7f23a2936"} Jul 21 13:01:16.689435 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.689400 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:16.709982 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.709936 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7349-140520293987360_8b20d9da-d0e3-423e-b3d6-c2220410b0d0/test-curl-7349-140520293987360/0.log" Jul 21 13:01:16.732920 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.732891 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:16.736317 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.736292 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7349-140520293987360"] Jul 21 13:01:16.829690 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.829602 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftrr7\" (UniqueName: \"kubernetes.io/projected/8b20d9da-d0e3-423e-b3d6-c2220410b0d0-kube-api-access-ftrr7\") pod \"8b20d9da-d0e3-423e-b3d6-c2220410b0d0\" (UID: \"8b20d9da-d0e3-423e-b3d6-c2220410b0d0\") " Jul 21 13:01:16.831752 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.831721 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b20d9da-d0e3-423e-b3d6-c2220410b0d0-kube-api-access-ftrr7" (OuterVolumeSpecName: "kube-api-access-ftrr7") pod "8b20d9da-d0e3-423e-b3d6-c2220410b0d0" (UID: "8b20d9da-d0e3-423e-b3d6-c2220410b0d0"). InnerVolumeSpecName "kube-api-access-ftrr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:01:16.857516 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.857482 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:16.858528 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.858504 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b20d9da-d0e3-423e-b3d6-c2220410b0d0" containerName="test-curl-7349-140520293987360" Jul 21 13:01:16.858528 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.858529 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b20d9da-d0e3-423e-b3d6-c2220410b0d0" containerName="test-curl-7349-140520293987360" Jul 21 13:01:16.859302 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.858788 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b20d9da-d0e3-423e-b3d6-c2220410b0d0" containerName="test-curl-7349-140520293987360" Jul 21 13:01:16.863243 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.863190 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:16.864604 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.864580 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:16.931585 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:16.931544 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftrr7\" (UniqueName: \"kubernetes.io/projected/8b20d9da-d0e3-423e-b3d6-c2220410b0d0-kube-api-access-ftrr7\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:17.032099 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.032059 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwp2q\" (UniqueName: \"kubernetes.io/projected/30a2b4d9-583f-4931-bb7f-91c0787b9009-kube-api-access-lwp2q\") pod \"test-curl-7349-140520293988368\" (UID: \"30a2b4d9-583f-4931-bb7f-91c0787b9009\") " pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:17.121409 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.121324 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b20d9da-d0e3-423e-b3d6-c2220410b0d0" path="/var/lib/kubelet/pods/8b20d9da-d0e3-423e-b3d6-c2220410b0d0/volumes" Jul 21 13:01:17.132718 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.132692 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwp2q\" (UniqueName: \"kubernetes.io/projected/30a2b4d9-583f-4931-bb7f-91c0787b9009-kube-api-access-lwp2q\") pod \"test-curl-7349-140520293988368\" (UID: \"30a2b4d9-583f-4931-bb7f-91c0787b9009\") " pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:17.141842 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.141815 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwp2q\" (UniqueName: \"kubernetes.io/projected/30a2b4d9-583f-4931-bb7f-91c0787b9009-kube-api-access-lwp2q\") pod \"test-curl-7349-140520293988368\" (UID: \"30a2b4d9-583f-4931-bb7f-91c0787b9009\") " pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:17.174227 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.174193 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:17.325320 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.325297 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:17.326240 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:17.326214 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30a2b4d9_583f_4931_bb7f_91c0787b9009.slice/crio-db33b7f5daf744d0a985e2346fabbbc336fb72f59804d451137a11b5fb758176 WatchSource:0}: Error finding container db33b7f5daf744d0a985e2346fabbbc336fb72f59804d451137a11b5fb758176: Status 404 returned error can't find the container with id db33b7f5daf744d0a985e2346fabbbc336fb72f59804d451137a11b5fb758176 Jul 21 13:01:17.546044 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.546016 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293987360" Jul 21 13:01:17.546240 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.546060 2550 scope.go:117] "RemoveContainer" containerID="fa4b4f7a61c0cdffbe760794055fde8c7576a860a9251cf0e17070d7f23a2936" Jul 21 13:01:17.547386 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:17.547302 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293988368" event={"ID":"30a2b4d9-583f-4931-bb7f-91c0787b9009","Type":"ContainerStarted","Data":"db33b7f5daf744d0a985e2346fabbbc336fb72f59804d451137a11b5fb758176"} Jul 21 13:01:18.557965 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:18.557931 2550 generic.go:356] "Generic (PLEG): container finished" podID="30a2b4d9-583f-4931-bb7f-91c0787b9009" containerID="8f7b3359ef713d719b1695fb204c15aec3f28359411a5bf0315256dfd9365c8a" exitCode=0 Jul 21 13:01:18.558349 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:18.557974 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293988368" event={"ID":"30a2b4d9-583f-4931-bb7f-91c0787b9009","Type":"ContainerDied","Data":"8f7b3359ef713d719b1695fb204c15aec3f28359411a5bf0315256dfd9365c8a"} Jul 21 13:01:19.709942 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.709917 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:19.730979 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.730950 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7349-140520293988368_30a2b4d9-583f-4931-bb7f-91c0787b9009/test-curl-7349-140520293988368/0.log" Jul 21 13:01:19.752999 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.752964 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:19.755894 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.755865 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwp2q\" (UniqueName: \"kubernetes.io/projected/30a2b4d9-583f-4931-bb7f-91c0787b9009-kube-api-access-lwp2q\") pod \"30a2b4d9-583f-4931-bb7f-91c0787b9009\" (UID: \"30a2b4d9-583f-4931-bb7f-91c0787b9009\") " Jul 21 13:01:19.756133 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.756115 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:19.757882 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.757860 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a2b4d9-583f-4931-bb7f-91c0787b9009-kube-api-access-lwp2q" (OuterVolumeSpecName: "kube-api-access-lwp2q") pod "30a2b4d9-583f-4931-bb7f-91c0787b9009" (UID: "30a2b4d9-583f-4931-bb7f-91c0787b9009"). InnerVolumeSpecName "kube-api-access-lwp2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:01:19.857669 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.857581 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwp2q\" (UniqueName: \"kubernetes.io/projected/30a2b4d9-583f-4931-bb7f-91c0787b9009-kube-api-access-lwp2q\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:19.864906 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.864879 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:19.865321 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.865309 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a2b4d9-583f-4931-bb7f-91c0787b9009" containerName="test-curl-7349-140520293988368" Jul 21 13:01:19.865373 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.865323 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a2b4d9-583f-4931-bb7f-91c0787b9009" containerName="test-curl-7349-140520293988368" Jul 21 13:01:19.865410 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.865404 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a2b4d9-583f-4931-bb7f-91c0787b9009" containerName="test-curl-7349-140520293988368" Jul 21 13:01:19.868568 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.868552 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:19.872726 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.872701 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:19.965913 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:19.959396 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcf9j\" (UniqueName: \"kubernetes.io/projected/d71de5a9-6691-44fc-910b-a1e623c157aa-kube-api-access-fcf9j\") pod \"test-curl-7349-140520293988368\" (UID: \"d71de5a9-6691-44fc-910b-a1e623c157aa\") " pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:20.060535 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.060501 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcf9j\" (UniqueName: \"kubernetes.io/projected/d71de5a9-6691-44fc-910b-a1e623c157aa-kube-api-access-fcf9j\") pod \"test-curl-7349-140520293988368\" (UID: \"d71de5a9-6691-44fc-910b-a1e623c157aa\") " pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:20.069511 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.069486 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcf9j\" (UniqueName: \"kubernetes.io/projected/d71de5a9-6691-44fc-910b-a1e623c157aa-kube-api-access-fcf9j\") pod \"test-curl-7349-140520293988368\" (UID: \"d71de5a9-6691-44fc-910b-a1e623c157aa\") " pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:20.178742 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.178715 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:20.330302 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.330269 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:20.331675 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:20.331647 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd71de5a9_6691_44fc_910b_a1e623c157aa.slice/crio-7cd71fd36efdc5cd2f64a5b1d8ab0e0c36be5557cd85c9e375e21993d72d79df WatchSource:0}: Error finding container 7cd71fd36efdc5cd2f64a5b1d8ab0e0c36be5557cd85c9e375e21993d72d79df: Status 404 returned error can't find the container with id 7cd71fd36efdc5cd2f64a5b1d8ab0e0c36be5557cd85c9e375e21993d72d79df Jul 21 13:01:20.571668 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.571515 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:20.571668 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.571517 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db33b7f5daf744d0a985e2346fabbbc336fb72f59804d451137a11b5fb758176" Jul 21 13:01:20.572806 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.572775 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293988368" event={"ID":"d71de5a9-6691-44fc-910b-a1e623c157aa","Type":"ContainerStarted","Data":"7cd71fd36efdc5cd2f64a5b1d8ab0e0c36be5557cd85c9e375e21993d72d79df"} Jul 21 13:01:20.583150 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:20.583125 2550 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7349-140520293988368" oldPodUID="30a2b4d9-583f-4931-bb7f-91c0787b9009" podUID="d71de5a9-6691-44fc-910b-a1e623c157aa" Jul 21 13:01:21.121581 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:21.121548 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a2b4d9-583f-4931-bb7f-91c0787b9009" path="/var/lib/kubelet/pods/30a2b4d9-583f-4931-bb7f-91c0787b9009/volumes" Jul 21 13:01:21.580529 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:21.580488 2550 generic.go:356] "Generic (PLEG): container finished" podID="d71de5a9-6691-44fc-910b-a1e623c157aa" containerID="60926173d9fc4de95be3ad37b9f163d7dd1d250f52503b4da2796acfd05cee55" exitCode=0 Jul 21 13:01:21.580712 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:21.580541 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7349-140520293988368" event={"ID":"d71de5a9-6691-44fc-910b-a1e623c157aa","Type":"ContainerDied","Data":"60926173d9fc4de95be3ad37b9f163d7dd1d250f52503b4da2796acfd05cee55"} Jul 21 13:01:22.756745 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.756719 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:22.777929 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.777903 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7349-140520293988368_d71de5a9-6691-44fc-910b-a1e623c157aa/test-curl-7349-140520293988368/0.log" Jul 21 13:01:22.797319 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.797282 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcf9j\" (UniqueName: \"kubernetes.io/projected/d71de5a9-6691-44fc-910b-a1e623c157aa-kube-api-access-fcf9j\") pod \"d71de5a9-6691-44fc-910b-a1e623c157aa\" (UID: \"d71de5a9-6691-44fc-910b-a1e623c157aa\") " Jul 21 13:01:22.799878 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.799847 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71de5a9-6691-44fc-910b-a1e623c157aa-kube-api-access-fcf9j" (OuterVolumeSpecName: "kube-api-access-fcf9j") pod "d71de5a9-6691-44fc-910b-a1e623c157aa" (UID: "d71de5a9-6691-44fc-910b-a1e623c157aa"). InnerVolumeSpecName "kube-api-access-fcf9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:01:22.801345 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.801320 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:22.806164 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.806138 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7349-140520293988368"] Jul 21 13:01:22.898507 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.898384 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcf9j\" (UniqueName: \"kubernetes.io/projected/d71de5a9-6691-44fc-910b-a1e623c157aa-kube-api-access-fcf9j\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:22.979881 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.979851 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll"] Jul 21 13:01:22.980317 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.980305 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d71de5a9-6691-44fc-910b-a1e623c157aa" containerName="test-curl-7349-140520293988368" Jul 21 13:01:22.980411 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.980318 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d71de5a9-6691-44fc-910b-a1e623c157aa" containerName="test-curl-7349-140520293988368" Jul 21 13:01:22.980411 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.980397 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d71de5a9-6691-44fc-910b-a1e623c157aa" containerName="test-curl-7349-140520293988368" Jul 21 13:01:22.983771 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.983755 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:22.986363 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.986345 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 13:01:22.991017 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:22.990991 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll"] Jul 21 13:01:23.099689 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.099656 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84719838-7db5-4db8-a422-058e1a75d1f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-782207-jw6ll\" (UID: \"84719838-7db5-4db8-a422-058e1a75d1f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:23.120576 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.120544 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d71de5a9-6691-44fc-910b-a1e623c157aa" path="/var/lib/kubelet/pods/d71de5a9-6691-44fc-910b-a1e623c157aa/volumes" Jul 21 13:01:23.200713 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.200682 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84719838-7db5-4db8-a422-058e1a75d1f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-782207-jw6ll\" (UID: \"84719838-7db5-4db8-a422-058e1a75d1f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:23.201259 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.201240 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84719838-7db5-4db8-a422-058e1a75d1f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-782207-jw6ll\" (UID: \"84719838-7db5-4db8-a422-058e1a75d1f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:23.296263 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.296228 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:23.593669 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.593573 2550 scope.go:117] "RemoveContainer" containerID="60926173d9fc4de95be3ad37b9f163d7dd1d250f52503b4da2796acfd05cee55" Jul 21 13:01:23.593669 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.593596 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7349-140520293988368" Jul 21 13:01:23.642573 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:23.642549 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll"] Jul 21 13:01:23.645033 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:23.645010 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84719838_7db5_4db8_a422_058e1a75d1f3.slice/crio-8df77424d47246485bf7efd5fdd13ecc90dbb720085f7820b2453313e9c035d4 WatchSource:0}: Error finding container 8df77424d47246485bf7efd5fdd13ecc90dbb720085f7820b2453313e9c035d4: Status 404 returned error can't find the container with id 8df77424d47246485bf7efd5fdd13ecc90dbb720085f7820b2453313e9c035d4 Jul 21 13:01:24.601100 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:24.601008 2550 generic.go:356] "Generic (PLEG): container finished" podID="84719838-7db5-4db8-a422-058e1a75d1f3" containerID="b26ab93f4641478b1d6c17a78c4ee7e3103f1fb0cf8b1c12650b0bd398dd21cf" exitCode=0 Jul 21 13:01:24.601520 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:24.601093 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" event={"ID":"84719838-7db5-4db8-a422-058e1a75d1f3","Type":"ContainerDied","Data":"b26ab93f4641478b1d6c17a78c4ee7e3103f1fb0cf8b1c12650b0bd398dd21cf"} Jul 21 13:01:24.601520 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:24.601193 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" event={"ID":"84719838-7db5-4db8-a422-058e1a75d1f3","Type":"ContainerStarted","Data":"8df77424d47246485bf7efd5fdd13ecc90dbb720085f7820b2453313e9c035d4"} Jul 21 13:01:25.752856 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:25.752827 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:25.826779 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:25.826745 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84719838-7db5-4db8-a422-058e1a75d1f3-maas-api-service-ca\") pod \"84719838-7db5-4db8-a422-058e1a75d1f3\" (UID: \"84719838-7db5-4db8-a422-058e1a75d1f3\") " Jul 21 13:01:25.827107 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:25.827083 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84719838-7db5-4db8-a422-058e1a75d1f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84719838-7db5-4db8-a422-058e1a75d1f3" (UID: "84719838-7db5-4db8-a422-058e1a75d1f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:01:25.928490 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:25.928463 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84719838-7db5-4db8-a422-058e1a75d1f3-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:26.614350 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:26.614322 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" Jul 21 13:01:26.614529 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:26.614357 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll" event={"ID":"84719838-7db5-4db8-a422-058e1a75d1f3","Type":"ContainerDied","Data":"8df77424d47246485bf7efd5fdd13ecc90dbb720085f7820b2453313e9c035d4"} Jul 21 13:01:26.614529 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:26.614400 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8df77424d47246485bf7efd5fdd13ecc90dbb720085f7820b2453313e9c035d4" Jul 21 13:01:33.605669 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:33.605637 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2"] Jul 21 13:01:33.607864 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:33.607841 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-782207-29743980-fvtz2"] Jul 21 13:01:35.120722 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:35.120693 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed77dcaf-e0c2-46f7-8ece-da47934963b3" path="/var/lib/kubelet/pods/ed77dcaf-e0c2-46f7-8ece-da47934963b3/volumes" Jul 21 13:01:38.095165 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:38.095121 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll"] Jul 21 13:01:38.097133 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:38.097109 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-782207-jw6ll"] Jul 21 13:01:39.121086 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:39.121054 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84719838-7db5-4db8-a422-058e1a75d1f3" path="/var/lib/kubelet/pods/84719838-7db5-4db8-a422-058e1a75d1f3/volumes" Jul 21 13:01:44.927177 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.927142 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9"] Jul 21 13:01:44.927665 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.927650 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84719838-7db5-4db8-a422-058e1a75d1f3" containerName="revoke-keys" Jul 21 13:01:44.927665 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.927666 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="84719838-7db5-4db8-a422-058e1a75d1f3" containerName="revoke-keys" Jul 21 13:01:44.927754 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.927735 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="84719838-7db5-4db8-a422-058e1a75d1f3" containerName="revoke-keys" Jul 21 13:01:44.932693 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.932675 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:44.935646 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.935616 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wrjps\"" Jul 21 13:01:44.937730 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:44.937708 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9"] Jul 21 13:01:45.017295 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:45.017251 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02114d70-ad1d-4dd7-a9df-e14b0faa95f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-48842b-28cs9\" (UID: \"02114d70-ad1d-4dd7-a9df-e14b0faa95f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:45.117896 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:45.117867 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02114d70-ad1d-4dd7-a9df-e14b0faa95f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-48842b-28cs9\" (UID: \"02114d70-ad1d-4dd7-a9df-e14b0faa95f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:45.118501 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:45.118476 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02114d70-ad1d-4dd7-a9df-e14b0faa95f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-48842b-28cs9\" (UID: \"02114d70-ad1d-4dd7-a9df-e14b0faa95f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:45.245499 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:45.245369 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:45.391264 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:45.391242 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9"] Jul 21 13:01:45.393374 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:01:45.393347 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02114d70_ad1d_4dd7_a9df_e14b0faa95f6.slice/crio-2cc9192ec722a7a5ff1e20eb5e0227b25a43e1d59d2276e1fb49e59b6743cd66 WatchSource:0}: Error finding container 2cc9192ec722a7a5ff1e20eb5e0227b25a43e1d59d2276e1fb49e59b6743cd66: Status 404 returned error can't find the container with id 2cc9192ec722a7a5ff1e20eb5e0227b25a43e1d59d2276e1fb49e59b6743cd66 Jul 21 13:01:45.725476 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:45.725442 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" event={"ID":"02114d70-ad1d-4dd7-a9df-e14b0faa95f6","Type":"ContainerStarted","Data":"2cc9192ec722a7a5ff1e20eb5e0227b25a43e1d59d2276e1fb49e59b6743cd66"} Jul 21 13:01:46.732890 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:46.732857 2550 generic.go:356] "Generic (PLEG): container finished" podID="02114d70-ad1d-4dd7-a9df-e14b0faa95f6" containerID="d7669719ea7214c123bf2aa6e9e3925fd8b5ca33b12100b22755d2e22a1a365f" exitCode=0 Jul 21 13:01:46.733235 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:46.732914 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" event={"ID":"02114d70-ad1d-4dd7-a9df-e14b0faa95f6","Type":"ContainerDied","Data":"d7669719ea7214c123bf2aa6e9e3925fd8b5ca33b12100b22755d2e22a1a365f"} Jul 21 13:01:47.881674 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:47.881649 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:48.047492 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:48.047371 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02114d70-ad1d-4dd7-a9df-e14b0faa95f6-maas-api-service-ca\") pod \"02114d70-ad1d-4dd7-a9df-e14b0faa95f6\" (UID: \"02114d70-ad1d-4dd7-a9df-e14b0faa95f6\") " Jul 21 13:01:48.047787 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:48.047766 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02114d70-ad1d-4dd7-a9df-e14b0faa95f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "02114d70-ad1d-4dd7-a9df-e14b0faa95f6" (UID: "02114d70-ad1d-4dd7-a9df-e14b0faa95f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:01:48.148528 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:48.148495 2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02114d70-ad1d-4dd7-a9df-e14b0faa95f6-maas-api-service-ca\") on node \"ip-10-0-136-33.ec2.internal\" DevicePath \"\"" Jul 21 13:01:48.745024 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:48.744994 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" Jul 21 13:01:48.745223 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:48.745032 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9" event={"ID":"02114d70-ad1d-4dd7-a9df-e14b0faa95f6","Type":"ContainerDied","Data":"2cc9192ec722a7a5ff1e20eb5e0227b25a43e1d59d2276e1fb49e59b6743cd66"} Jul 21 13:01:48.745223 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:48.745102 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cc9192ec722a7a5ff1e20eb5e0227b25a43e1d59d2276e1fb49e59b6743cd66" Jul 21 13:01:55.557705 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:55.557674 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d"] Jul 21 13:01:55.563892 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:55.563865 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-48842b-29743980-47f2d"] Jul 21 13:01:57.120934 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:01:57.120900 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21af0ca4-55f4-4c6e-af07-928d9962929a" path="/var/lib/kubelet/pods/21af0ca4-55f4-4c6e-af07-928d9962929a/volumes" Jul 21 13:02:00.011752 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:00.011653 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9"] Jul 21 13:02:00.013934 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:00.013914 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-48842b-28cs9"] Jul 21 13:02:01.121583 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:01.121550 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02114d70-ad1d-4dd7-a9df-e14b0faa95f6" path="/var/lib/kubelet/pods/02114d70-ad1d-4dd7-a9df-e14b0faa95f6/volumes" Jul 21 13:02:07.163558 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:07.163530 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75674d865c-cw8r6_f83c5620-4886-4384-a8e8-66aff32b36a8/authorino/0.log" Jul 21 13:02:12.476379 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:12.476347 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xcq5l_2af2e541-15f0-4b30-b60a-2891733736b4/manager/0.log" Jul 21 13:02:12.590076 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:12.590042 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7lskl_19c55814-d746-4eeb-bd8b-815d7aff4ff9/manager/0.log" Jul 21 13:02:12.843013 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:12.842940 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bstq7_02182868-a2d3-489f-a93f-9d33d730fe57/server/0.log" Jul 21 13:02:12.960160 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:12.960131 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c5l8l_2f37db97-d0e9-434f-964f-26b04ab3b139/manager/0.log" Jul 21 13:02:13.374512 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:13.374479 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-rt6rv_95006fc7-bc4a-499a-979e-ec8d0748ec43/manager/0.log" Jul 21 13:02:13.382141 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:13.382121 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-rt6rv_95006fc7-bc4a-499a-979e-ec8d0748ec43/copy-manifests/0.log" Jul 21 13:02:13.712296 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:13.712264 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74d896f5b9-hvctb_9a9c2ffe-3280-452e-8426-14f02aa0cdc7/maas-api/0.log" Jul 21 13:02:13.960447 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:13.960397 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743965-vcmq2_75cea34b-85a9-4115-b231-1b9afa788642/cleanup/0.log" Jul 21 13:02:14.075184 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:14.075100 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743980-pg58b_55b83db6-3e34-4601-9e85-5309136eba00/cleanup/0.log" Jul 21 13:02:14.188960 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:14.188933 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hw7bn_2cf943c3-7c52-42d0-98ae-ae4ce9a6e62a/postgres/0.log" Jul 21 13:02:15.459380 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:15.459353 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75674d865c-cw8r6_f83c5620-4886-4384-a8e8-66aff32b36a8/authorino/0.log" Jul 21 13:02:15.597248 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:15.597217 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z774d_4c6589f7-d18d-4d0e-8072-eab93d368549/manager/0.log" Jul 21 13:02:15.710331 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:15.710254 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gbzl4_8e6755aa-5c16-42ec-ab43-ccc580fa6bc4/manager/0.log" Jul 21 13:02:15.940765 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:15.940737 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4lxbd_841d08d0-4a98-40a4-b817-92ee60119023/registry-server/0.log" Jul 21 13:02:16.294441 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:16.294389 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ljkqs_33d38dd6-fd73-4b64-a223-5c2b137241d3/manager/0.log" Jul 21 13:02:16.652391 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:16.652320 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wq799_e61b7390-f5e5-4c3c-95cb-2362d817361e/istio-proxy/0.log" Jul 21 13:02:17.046601 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:17.046575 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc49b7cdf-vtfmx_4531b066-13dd-4ad0-9663-edc34afb7245/kube-auth-proxy/0.log" Jul 21 13:02:17.172943 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:17.172914 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/1.log" Jul 21 13:02:17.539621 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:17.539593 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568fbddcbd-x8pzc_eeaacab5-50b2-4ea4-8f6b-29eae71b790f/router/0.log" Jul 21 13:02:17.909474 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:17.909383 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5w8nr_5a819487-9a20-4cc0-b836-745ef53b38b0/main/0.log" Jul 21 13:02:18.256702 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:18.256660 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn745l_646b691b-252a-4ee6-a6ad-50856937ba55/main/0.log" Jul 21 13:02:24.143982 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.143952 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqg59/must-gather-2h48j"] Jul 21 13:02:24.144405 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.144392 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02114d70-ad1d-4dd7-a9df-e14b0faa95f6" containerName="revoke-keys" Jul 21 13:02:24.144405 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.144405 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="02114d70-ad1d-4dd7-a9df-e14b0faa95f6" containerName="revoke-keys" Jul 21 13:02:24.144523 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.144517 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="02114d70-ad1d-4dd7-a9df-e14b0faa95f6" containerName="revoke-keys" Jul 21 13:02:24.147816 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.147792 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.150969 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.150950 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqg59\"/\"kube-root-ca.crt\"" Jul 21 13:02:24.151076 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.150950 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqg59\"/\"openshift-service-ca.crt\"" Jul 21 13:02:24.166925 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.166898 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqg59/must-gather-2h48j"] Jul 21 13:02:24.304362 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.304329 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2697f1ab-bfc8-4925-9a45-60d70f61ae8c-must-gather-output\") pod \"must-gather-2h48j\" (UID: \"2697f1ab-bfc8-4925-9a45-60d70f61ae8c\") " pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.304545 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.304417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gkf2\" (UniqueName: \"kubernetes.io/projected/2697f1ab-bfc8-4925-9a45-60d70f61ae8c-kube-api-access-2gkf2\") pod \"must-gather-2h48j\" (UID: \"2697f1ab-bfc8-4925-9a45-60d70f61ae8c\") " pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.406028 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.405938 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2697f1ab-bfc8-4925-9a45-60d70f61ae8c-must-gather-output\") pod \"must-gather-2h48j\" (UID: \"2697f1ab-bfc8-4925-9a45-60d70f61ae8c\") " pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.406028 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.406020 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gkf2\" (UniqueName: \"kubernetes.io/projected/2697f1ab-bfc8-4925-9a45-60d70f61ae8c-kube-api-access-2gkf2\") pod \"must-gather-2h48j\" (UID: \"2697f1ab-bfc8-4925-9a45-60d70f61ae8c\") " pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.412697 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.406276 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2697f1ab-bfc8-4925-9a45-60d70f61ae8c-must-gather-output\") pod \"must-gather-2h48j\" (UID: \"2697f1ab-bfc8-4925-9a45-60d70f61ae8c\") " pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.429197 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.429166 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gkf2\" (UniqueName: \"kubernetes.io/projected/2697f1ab-bfc8-4925-9a45-60d70f61ae8c-kube-api-access-2gkf2\") pod \"must-gather-2h48j\" (UID: \"2697f1ab-bfc8-4925-9a45-60d70f61ae8c\") " pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.457045 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.457011 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqg59/must-gather-2h48j" Jul 21 13:02:24.620570 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.620543 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqg59/must-gather-2h48j"] Jul 21 13:02:24.622709 ip-10-0-136-33 kubenswrapper[2550]: W0721 13:02:24.622687 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2697f1ab_bfc8_4925_9a45_60d70f61ae8c.slice/crio-d62d770224a7c12dacd0916a317bb528e2f8ff4e8187475dbe10f9408f6af276 WatchSource:0}: Error finding container d62d770224a7c12dacd0916a317bb528e2f8ff4e8187475dbe10f9408f6af276: Status 404 returned error can't find the container with id d62d770224a7c12dacd0916a317bb528e2f8ff4e8187475dbe10f9408f6af276 Jul 21 13:02:24.955494 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:24.955458 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/must-gather-2h48j" event={"ID":"2697f1ab-bfc8-4925-9a45-60d70f61ae8c","Type":"ContainerStarted","Data":"d62d770224a7c12dacd0916a317bb528e2f8ff4e8187475dbe10f9408f6af276"} Jul 21 13:02:26.973183 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:26.973141 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/must-gather-2h48j" event={"ID":"2697f1ab-bfc8-4925-9a45-60d70f61ae8c","Type":"ContainerStarted","Data":"9b9eefeff49774a429be81b6d27dbab7652cae3cfc25c8baecae53e4f7b5d32b"} Jul 21 13:02:26.973183 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:26.973197 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/must-gather-2h48j" event={"ID":"2697f1ab-bfc8-4925-9a45-60d70f61ae8c","Type":"ContainerStarted","Data":"6735b08c90597bca1f3f4cee87274ec450e061ffe535f6e2408ebcde3915178a"} Jul 21 13:02:26.989279 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:26.989225 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqg59/must-gather-2h48j" podStartSLOduration=1.676513437 podStartE2EDuration="2.989204549s" totalImagesPullingTime="1.312691112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:02:26.988606508 +0000 UTC m=+3080.482435496" watchObservedRunningTime="2026-07-21 13:02:26.989204549 +0000 UTC m=+3080.483033539" Jul 21 13:02:27.600320 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:27.600269 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rrb6l_a4066c5a-25b5-4a23-80f7-ef95d3516d70/global-pull-secret-syncer/0.log" Jul 21 13:02:27.871840 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:27.871757 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m7bb6_4b8bfc2d-09ba-415b-82f2-bc35e42c93ff/konnectivity-agent/0.log" Jul 21 13:02:27.963714 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:27.963675 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-33.ec2.internal_8dc81c72d91b2f4557f0246b9d50dc4e/haproxy/0.log" Jul 21 13:02:33.124447 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:33.123755 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75674d865c-cw8r6_f83c5620-4886-4384-a8e8-66aff32b36a8/authorino/0.log" Jul 21 13:02:33.163152 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:33.163101 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z774d_4c6589f7-d18d-4d0e-8072-eab93d368549/manager/0.log" Jul 21 13:02:33.192107 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:33.192078 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gbzl4_8e6755aa-5c16-42ec-ab43-ccc580fa6bc4/manager/0.log" Jul 21 13:02:33.281250 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:33.281144 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4lxbd_841d08d0-4a98-40a4-b817-92ee60119023/registry-server/0.log" Jul 21 13:02:33.606999 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:33.606960 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ljkqs_33d38dd6-fd73-4b64-a223-5c2b137241d3/manager/0.log" Jul 21 13:02:35.535336 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.535305 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-7l28l_41a6c18f-f386-4739-8fa7-2acdfdf46223/cluster-monitoring-operator/0.log" Jul 21 13:02:35.639992 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.639965 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79dc965d9c-cvbd8_6bc3193f-d43b-4f80-a687-d34552366a92/metrics-server/0.log" Jul 21 13:02:35.707592 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.707560 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2j5wl_ec2db094-7e6c-4e30-a5fb-94cb23b33b8f/node-exporter/0.log" Jul 21 13:02:35.736953 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.736920 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2j5wl_ec2db094-7e6c-4e30-a5fb-94cb23b33b8f/kube-rbac-proxy/0.log" Jul 21 13:02:35.764030 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.764006 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2j5wl_ec2db094-7e6c-4e30-a5fb-94cb23b33b8f/init-textfile/0.log" Jul 21 13:02:35.950007 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.949978 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-dhlfb_2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7/kube-rbac-proxy-main/0.log" Jul 21 13:02:35.974737 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.974714 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-dhlfb_2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7/kube-rbac-proxy-self/0.log" Jul 21 13:02:35.999180 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:35.999134 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-dhlfb_2c0b72d9-c5f6-4edf-8b4d-baa9b03381e7/openshift-state-metrics/0.log" Jul 21 13:02:36.317637 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.317314 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b784ddd67-dhlph_b50e76da-c961-48c0-bcfc-450e5a4bba97/telemeter-client/0.log" Jul 21 13:02:36.341009 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.340978 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b784ddd67-dhlph_b50e76da-c961-48c0-bcfc-450e5a4bba97/reload/0.log" Jul 21 13:02:36.366001 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.365971 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b784ddd67-dhlph_b50e76da-c961-48c0-bcfc-450e5a4bba97/kube-rbac-proxy/0.log" Jul 21 13:02:36.402065 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.402037 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f45cdfd54-dqxv4_f5b0ad4d-1dad-455c-a0bd-6ea6c9724838/thanos-query/0.log" Jul 21 13:02:36.425390 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.425346 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f45cdfd54-dqxv4_f5b0ad4d-1dad-455c-a0bd-6ea6c9724838/kube-rbac-proxy-web/0.log" Jul 21 13:02:36.449533 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.449502 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f45cdfd54-dqxv4_f5b0ad4d-1dad-455c-a0bd-6ea6c9724838/kube-rbac-proxy/0.log" Jul 21 13:02:36.504296 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.504260 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f45cdfd54-dqxv4_f5b0ad4d-1dad-455c-a0bd-6ea6c9724838/prom-label-proxy/0.log" Jul 21 13:02:36.526046 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.526015 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f45cdfd54-dqxv4_f5b0ad4d-1dad-455c-a0bd-6ea6c9724838/kube-rbac-proxy-rules/0.log" Jul 21 13:02:36.549682 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:36.549636 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f45cdfd54-dqxv4_f5b0ad4d-1dad-455c-a0bd-6ea6c9724838/kube-rbac-proxy-metrics/0.log" Jul 21 13:02:37.720193 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.720165 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-22p7z_133dbdd2-a362-4d8c-adfd-fd1dca5ff65d/networking-console-plugin/0.log" Jul 21 13:02:37.797888 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.797847 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d"] Jul 21 13:02:37.806519 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.806486 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.809857 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.809661 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nqg59\"/\"default-dockercfg-6wm6t\"" Jul 21 13:02:37.817042 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.817016 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d"] Jul 21 13:02:37.866357 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.866181 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-podres\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.866357 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.866319 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-sys\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.866357 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.866356 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-lib-modules\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.866706 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.866521 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rt5p\" (UniqueName: \"kubernetes.io/projected/a49a6f19-05bc-49e9-b14a-04f7c9406adb-kube-api-access-5rt5p\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.866706 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.866588 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-proc\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967339 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967306 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rt5p\" (UniqueName: \"kubernetes.io/projected/a49a6f19-05bc-49e9-b14a-04f7c9406adb-kube-api-access-5rt5p\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967567 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967374 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-proc\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967567 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967455 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-podres\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967567 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967545 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-sys\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967567 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967556 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-proc\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967804 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967575 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-lib-modules\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967804 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967776 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-podres\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967804 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967787 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-sys\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.967804 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.967779 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a49a6f19-05bc-49e9-b14a-04f7c9406adb-lib-modules\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:37.976645 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:37.976569 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rt5p\" (UniqueName: \"kubernetes.io/projected/a49a6f19-05bc-49e9-b14a-04f7c9406adb-kube-api-access-5rt5p\") pod \"perf-node-gather-daemonset-c6c6d\" (UID: \"a49a6f19-05bc-49e9-b14a-04f7c9406adb\") " pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:38.129792 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:38.129745 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:38.329450 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:38.327048 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d"] Jul 21 13:02:38.412724 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:38.412694 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/1.log" Jul 21 13:02:38.421636 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:38.421607 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dvskp_4fc860f4-873b-4e1e-8e3c-503f24ab6d86/console-operator/2.log" Jul 21 13:02:38.966329 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:38.966257 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-797b978cdd-9wfzz_fb8ef86b-1dff-4409-b607-d84c695fb765/console/0.log" Jul 21 13:02:39.007188 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:39.007156 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-thrrb_f9b9baf9-31e6-42c8-b1fa-fa4f357396a0/download-server/0.log" Jul 21 13:02:39.101647 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:39.101607 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" event={"ID":"a49a6f19-05bc-49e9-b14a-04f7c9406adb","Type":"ContainerStarted","Data":"7a1da150d7f4d8257ab2d1244ed0314db56b119d5d769a6041e5905e2c60f18f"} Jul 21 13:02:39.101937 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:39.101911 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" event={"ID":"a49a6f19-05bc-49e9-b14a-04f7c9406adb","Type":"ContainerStarted","Data":"91615da49f853163af41d12e4228e5739a917a339f905ab874b3984e2ed7901d"} Jul 21 13:02:39.102132 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:39.102115 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:39.124030 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:39.123970 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" podStartSLOduration=2.123948318 podStartE2EDuration="2.123948318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:02:39.120887191 +0000 UTC m=+3092.614716186" watchObservedRunningTime="2026-07-21 13:02:39.123948318 +0000 UTC m=+3092.617777314" Jul 21 13:02:39.628800 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:39.628763 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-8hg6z_67b67449-f536-4272-bb03-f8224c8e196e/volume-data-source-validator/0.log" Jul 21 13:02:40.731250 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:40.731222 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nrn6d_396d16ec-f45b-427e-846a-68ee756f846c/dns/0.log" Jul 21 13:02:40.759264 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:40.759238 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nrn6d_396d16ec-f45b-427e-846a-68ee756f846c/kube-rbac-proxy/0.log" Jul 21 13:02:40.886881 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:40.886858 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pb5qj_502316a9-ade5-4bf4-b1f8-f6868b87987b/dns-node-resolver/0.log" Jul 21 13:02:41.561742 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:41.561702 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wxzfm_a5f7f769-70f8-4cfe-a810-ff6d3e476ce4/node-ca/0.log" Jul 21 13:02:42.524751 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:42.524715 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wq799_e61b7390-f5e5-4c3c-95cb-2362d817361e/istio-proxy/0.log" Jul 21 13:02:42.814374 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:42.814298 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc49b7cdf-vtfmx_4531b066-13dd-4ad0-9663-edc34afb7245/kube-auth-proxy/0.log" Jul 21 13:02:42.844919 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:42.844887 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/0.log" Jul 21 13:02:42.878821 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:42.878788 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/1.log" Jul 21 13:02:42.991072 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:42.991039 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568fbddcbd-x8pzc_eeaacab5-50b2-4ea4-8f6b-29eae71b790f/router/0.log" Jul 21 13:02:43.722955 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:43.722916 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fl9m4_317bf622-f052-40df-be20-3269ccdf1cda/serve-healthcheck-canary/0.log" Jul 21 13:02:44.249376 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:44.249341 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-zsqww_a380abd7-3ed2-4c49-b8b7-18431a1f7469/insights-operator/0.log" Jul 21 13:02:44.250384 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:44.250361 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-zsqww_a380abd7-3ed2-4c49-b8b7-18431a1f7469/insights-operator/1.log" Jul 21 13:02:44.272346 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:44.272317 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4mjkh_1f1663b6-40b5-4515-b534-041eb295876f/kube-rbac-proxy/0.log" Jul 21 13:02:44.293579 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:44.293549 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4mjkh_1f1663b6-40b5-4515-b534-041eb295876f/exporter/0.log" Jul 21 13:02:44.317516 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:44.317478 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4mjkh_1f1663b6-40b5-4515-b534-041eb295876f/extractor/0.log" Jul 21 13:02:45.125181 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:45.124392 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nqg59/perf-node-gather-daemonset-c6c6d" Jul 21 13:02:47.127825 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:47.127773 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xcq5l_2af2e541-15f0-4b30-b60a-2891733736b4/manager/0.log" Jul 21 13:02:47.405574 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:47.405534 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7lskl_19c55814-d746-4eeb-bd8b-815d7aff4ff9/manager/0.log" Jul 21 13:02:47.635418 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:47.635365 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bstq7_02182868-a2d3-489f-a93f-9d33d730fe57/server/0.log" Jul 21 13:02:47.733774 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:47.733736 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c5l8l_2f37db97-d0e9-434f-964f-26b04ab3b139/manager/0.log" Jul 21 13:02:47.939324 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:47.939254 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-rt6rv_95006fc7-bc4a-499a-979e-ec8d0748ec43/manager/0.log" Jul 21 13:02:47.962924 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:47.962891 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58f85c6b8d-rt6rv_95006fc7-bc4a-499a-979e-ec8d0748ec43/copy-manifests/0.log" Jul 21 13:02:48.774692 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:48.774646 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wq799_e61b7390-f5e5-4c3c-95cb-2362d817361e/istio-proxy/0.log" Jul 21 13:02:49.043502 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:49.043395 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc49b7cdf-vtfmx_4531b066-13dd-4ad0-9663-edc34afb7245/kube-auth-proxy/0.log" Jul 21 13:02:49.072327 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:49.072268 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/0.log" Jul 21 13:02:49.107925 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:49.107889 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/1.log" Jul 21 13:02:49.177280 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:49.177250 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568fbddcbd-x8pzc_eeaacab5-50b2-4ea4-8f6b-29eae71b790f/router/0.log" Jul 21 13:02:49.316193 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:49.316073 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jg9xv_611e5e95-6cdf-4ecd-a483-dba117a82847/openshift-lws-operator/0.log" Jul 21 13:02:50.426719 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:50.426693 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wq799_e61b7390-f5e5-4c3c-95cb-2362d817361e/istio-proxy/0.log" Jul 21 13:02:50.677820 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:50.677735 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fc49b7cdf-vtfmx_4531b066-13dd-4ad0-9663-edc34afb7245/kube-auth-proxy/0.log" Jul 21 13:02:50.699171 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:50.699135 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/0.log" Jul 21 13:02:50.735181 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:50.735144 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-glp4d_9cff849f-9b57-4685-85a2-6b0a707a6670/istio-proxy/1.log" Jul 21 13:02:50.807523 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:50.807494 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568fbddcbd-x8pzc_eeaacab5-50b2-4ea4-8f6b-29eae71b790f/router/0.log" Jul 21 13:02:54.658914 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:54.658888 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-rgkhb_7135034a-0b9e-44b6-9619-93b88cf3a501/kube-storage-version-migrator-operator/1.log" Jul 21 13:02:54.659850 ip-10-0-136-33 kubenswrapper[2550]: I0721 13:02:54.659825 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-rgkhb_7135034a-0b9e-44b6-9619-93b88cf3a501/kube-storage-version-migrator-operator/0.log"